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HOW IMPORTANT IS YOUR DATA? 


Years of family photos. Your entire music 
and movie collection. Office documents 
you've put hours of work into. Backups for 
every computer you own. We ask again, how 
important is your data? 


NOW IMAGINE LOSING IT ALL 


Losing one bit - that’s all it takes. One single bit, and 
your file is gone. 


The worst part? You won't know until you | 
absolutely need that file again. Example of one-bit corruption 


THE SOLUTION 


The Mini boasts these state-of-the- 


The FreeNAS Mini has emerged as the clear choice to 
art features: 


Save your digital life. No other NAS in its class offers 


i ry and ZFS bitr 
ECC (error correcting code) memory and ZFS bitrot ee ee ee 


protection to ensure data always reaches disk . Up to 16TB of storage capacity 
without corruption and never degrades over time. - 16GB of ECC memory (with the option to upgrade 
to 32GB) 


« 2x 1 Gigabit network controllers 
No other NAS combines the inherent data integrity : Ramotemansuementoort (EN 


and security of the ZFS filesystem with fast on-disk - Tool-less design; hot swappable drive trays 
encryption. No other NAS provides comparable power oRiSe NPS ictal emanecomngurey 

and flexibility. The FreeNAS Mini is, hands-down, the 
best home and small office storage appliance you can 
buy on the market. When it comes to saving your 
important data, there simply is no other solution. 
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FREENAS 


CERTIFIED 
STORAGE 


With over six million downloads, 
FreeNAS is undisputedly the most 
popular storage operating system 
in the world. 


Sure, you could build your own FreeNAS system: 
research every hardware option, order all the 

parts, wait for everything to ship and arrive, vent at 
customer service because it hasn't, and finally build it 
yourself while hoping everything fits - only to install 
the software and discover that the system you spent 
days agonizing over isn’t even compatible. Or... 


MAKE IT EASY ON YOURSELF 


As the sponsors and lead developers of the FreeNAS 
project, ixsystems has combined over 20 years of 
hardware experience with our FreeNAS expertise to 
bring you FreeNAS Certified Storage. We make it 
easy to enjoy all the benefits of FreeNAS without 
the headache of building, setting up, configuring, 
and supporting it yourself. As one of the leaders in 
the storage industry, you know that you're getting the 
best combination of hardware designed for optimal 
performance with FreeNAS. 


Every FreeNAS server we ship is... 


» Custom built and optimized for your use case 

» Installed, configured, tested, and guaranteed to work out 
of the box 

» Supported by the Silicon Valley team that designed and 
built it 

» Backed by a 3 years parts and labor limited warranty 
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As one of the leaders in the storage industry, you 
know that you're getting the best combination 

of hardware designed for optimal performance 

with FreeNAS. Contact us today for a FREE Risk 
Elimination Consultation with one of our FreeNAS 
experts. Remember, every purchase directly supports 
the FreeNAS project so we can continue adding 
features and improvements to the software for years 
to come. And really - why would you buy a FreeNAS 
server from anyone else? 


FreeNAS 1U 

- Intel® Xeon® Processor E3-1200v2 Family 

- Up to 16TB of storage capacity 

- 16GB ECC memory (upgradable to 32GB) 

« 2x 10/100/1000 Gigabit Ethernet controllers 
- Redundant power supply 


FreeNAS 2U 
- 2x Intel® Xeon® Processors E5-2600v2 Family 
- Up to 48TB of storage capacity 
- 32GB ECC memory (upgradable to 128GB) 
« 4x 1GbE Network interface (Onboard) - 
(Upgradable to 2 x 10 Gigabit Interface) 
« Redundant Power Supply 


http://www.iXsystems.com/storage/freenas-certified-storage/ 
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EDITOR’S WORD 


Dear Readers, 


# ce to meet you again. Now you are going to read 
: the next issue from BSD magazine. You have the 


chance to walk through the installation and the basic 
contiguration of Postfix, one of the most popular SMTP servers, 
and SpamAssassin, which will be used for basic e-mail filtering. 


What is more, our experts will show you that most of the 
companies stick to stable/release versions with only security fixes. 
Indeed, if your applications rely on specific API/ABI versions, it is 
better to keep on doing it, but others run experimental branches. 
You will learn more from our article written by David Carlier. 


Finally, you may find interest in the article from the Technologies 
section. The article gives you more insight into industry practices. 


We would like to express our gratitude to our experts who 


contributed to this publication and invite others to cooperate with 
our magazine. 


Enjoy reading, 
Ewa & the BSD team 
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IN BUSINESS 


FreeNAS 
in an Enterprise Environment 


By the time you're reading this, FreeNAS has been downloaded 
more than 5.5 million times. For home users, it’s become an 
indispensable part of their daily lives, akin to the DVR. 
Meanwhile, all over the world, thousands of businesses 
universities, and government departments use FreeNAS to 
build effective storage solutions in myriad applications 
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BSD -CURRENT is Usable Daily Ss 


David Carlier 

Running the development branch of a *BSD daily might sound 
scary. Indeed, this is basically the experimentations’ land and 
this use case seems to apply only to BSD developers — the 
internal APIs might suddenly change because they need to, 
some bugs can be fixed. But some new ones can be introduced 
without notice ... Although that, in general, the community is 
quite reactive and fixes them fairly quickly. David will explain the 
reasons of using what is called the -CURRENT branches. 


Installing the E-mail Servers 
and the Webmail Interface 
Ivan Voras 

The goal of this article is to walk you through the installation and 
the basic configuration of Postfix, one of the most popular SMTP 
servers, and SpamAssassin, which will be used for basic e-mail 
filtering. 
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The Basics of The GDB Debugger -O 


Carlos Neira 

To be able to inspect a program more easily, we need to have the 
symbol table available for the program we intend to debug. This 
is accomplished using the —g flag of the compiler we are going 
to use (we could also debug it without the —g flag but it is really 
cumbersome sometimes). In our case, we will use FreeBSD 10 
as the platform and the clang compiler that comes with it. 
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A Complete Guide to FreeNAS 
Hardware Design, Part |: 
Purpose and Best Practices 
Josh Paetzel 

A guide to selecting and building FreeNAS hardware, written 
by the FreeNAS Team, is long past overdue by now. For that, 
we apologize. The issue was the depth and complexity of the 
subject, as you will see by the extensive nature of this four part 
guide, due to the variety of ways FreeNAS can be utilized. 
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Column 


Has the technology sector finally slid into 
the realm of used car salesmen, lawyers 

and ambulance chasers? 38 
Rob Somerville 


Useful Technologies 


Information Security Analytics Finding 
Security Insights, Patterns, 

and Anomalies in Big Data. 
Simulations and Security Processes 
Mark Ryan Talabis, Robert McPherson, 
Inez Miyamoto and Jason L. Martin 
Information Security Analytics gives you insights into the practice 
of analytics and, more importantly, how you can utilize analytic 
techniques to identify trends and outliers that may not be possible 
to identify using traditional security analysis techniques. 
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*BSD CORNER 


BSD - 
is Usable Daily 


CURRENT 


Running the development branch of a *BSD daily might 
sound scary. Indeed, this is basically the experimentations’ 
land and this use case seems to apply only to BSD developers 
— the internal APIs might suddenly change because they 
need to, some bugs can be fixed, some new ones can 

be introduced without notice (although in general, the 
community is quite reactive and fixes them fairly quickly). 
lam going to talk about the BSDs | know and use the most 
and I'll explain the reasons for using the -CURRENT branches. 


O 


latest version possi- 
ble of clang because | 
am following the com- 
ing of some expected 
features, like OpenMP 
support and_ sanitiz- 
ers support, because 
of the compilation ef- 
fectiveness improve- 
ments, and so on. As 
| often use virtualized 
environments, having 
the latest bhyve fea- 
tures iS a very good 
point. From a develop- 
er point of view, hav- 
ing new syscalls like 
explicit_bzero (which 
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ne of the main reasons which | use -CURRENT 
branches is simply having the latest innovations. 
In the case of FreeBSD, | like having the very 


FreeBSD 11.@-CURRENT (IRONFIST) #3 1279098: Sat Feb 21 09:26:20 GMT 2015 


can be preferred in place of memset for some use cas- 
es, avoiding the potential compiler optimization ...), ppoll 
for the Linux emulation layer are beneficial. Casperd pro- 


Welcome to FreeBSD! 


Release Notes, Errata: https: //www.FreeBSD.org/releases/ 

Security Advisories:  https://www.FreeBSD.org/security/ 

FreeBSD Handbook: https: //www. FreeBSD. org/handbook/ 

FreeBSD FAQ: https: //www. FreeBSD. org/faq/ 

Questions List: https://lists.FreeBSD.org/mailman/ listinfo/freebsd-—questions/ 
FreeBSD Forums: https://forums.FreeBSD.org/ 


Documents installed with the system are in the /usr/local/share/doc/freebsd/ 
directory, or can be installed later with: pkg install en-freebsd-doc 
For other languages, replace "en" with a language code like de or fr. 


Show the version of FreeBSD installed: freebsd-version ; uname -a 
Please include that output and any error messages when posting questions. 
Introduction to manual pages: man man 

FreeBSD directory Layout: man hier 


Edit /etc/motd to change this login announcement. 


Figure 1. FreeBSD CURRENT 
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vides some services not available in capsicum’s capabili- 
ties mode and can be seen as a proxy, for example, for 
DNS resolution. 


For OpenBSD, having the latest relayd/httpd features 
interests me (i.e., | run a custom version of relayd which 
produces some additional custom HTTP headers). | ap- 
preciate their “backward compatibility breaking fearless 
for the better good” approach (the recent change in ran- 
dom C API, for example, could confirm it). Indeed since 
the 5.6, the static Position Independent Executable sup- 
port for base system binaries was added, the legacy de- 
terministic rand C API was strongly updated, and so on... 


| recently retried NetBSD, with LLVM/clang in base fol- 
lowing their willingness to move towards it. After some 
days of usage, | noticed a general small performance drop 
(one of my custom applications got something like 5/10 
percent of difference) but it is a generally well Known prob- 
lem with clang; it is improving through the releases. 


Lastly, DragonflyBSD recently brought GCC 5.0 in base 
(with a bunch of new sanitizations flags, in addition to the 
OpenMP 4.0 specifications support). Also more generally, 
a lot of effort is made in the graphic stack. Having the last 
fixes for Hammer 1 filesystem is worthwhile (i.e Hammer2 
is still not production ready). 


=== Rebuilding the temporary build tree 


>>> Stage 1.1: legacy release compatibility shims 


==> stage 1.2: bootstrap tools 


===> Lib/clang/libllvmsupport (obj,depend,all, install) 
===> Lib/clang/libllvmtablegen (obj,depend,all, install) 
===> uSr.bin/clang/tblgen (obj,depend,all, install) 

===> usr.bin/clang/clang-tblgen (obj,depend,all, install) 
===> kerberos5/tools/make-roken (obj,depend,all, install) 
===> kerberos5/lib/libroken (obj,depend,all, install) 
===> kerberos5/lib/libvers (obj,depend,all, install) 

===> kerberos5/tools/asni_compile (obj,depend,all, install) 
yacc: 4 shift/reduce conflicts. 

===> kerberos5/tools/slc (obj,depend,all, install) 

===> usr.bin/compile_et (obj,depend,all, install) 

===> games/fortune/strfile (obj,depend,all, install) 

===> gnu/usr.bin/gperf (obj,depend,all, install) 

===> gnu/usr.bin/groff (obj,depend,all, install) 


Figure 2. Recompiling FreeBSD... The time needed is fairly variable 
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BSD -CURRENT... 


One of the downsides of running current is if you’re us- 
ing a desktop environment or more generally the ports 
system. In general, when a significant change in the base 
system occurs, it is recommended to rebuild all the ports 
afterwards. The time needed to do so could be poten- 
tially quite important, especially with software like KDE, 
Gnome 3, etc. It is a point to weigh well ... 


For FreeBSD -CURRENT, | very rarely run a desk- 
top. | prefer to use the whole potential CPU/memory for 
compiling the system instead. Also, the fact that | enable 
a significant amount of debugging kernel options which 
slow down the general performance (like WITNESS 
(to detect potential deadlocks) / INVARIANTS (which add 
more kernel level’s assertion) flags) stops me from con- 
sidering it. Those specific options are only useful for de- 
velopers or beta testers though. It is advised to disable 
them otherwise. 


In the case of OpenBSD -CURRENT, | run time in time 
the base cwn which is very light and xorg (called xeno- 
cara) Is not in the ports but in the base system, that makes 
those updates easier. In addition, | enable MALLOC _ 
STATS, hence allowing the D flag for MALLOC_OPTIONS 
for debugging purposes with the cost at a performance hit. 
Again, this last one is not recommended if you are not 
a developer. 


OpenBSD 5.7-beta (GENERIC.MP) #38: Fri Feb 20 20:19:23 GMT 28615 


Welcome to OpenBSD: The proactively secure Unix-like operating system. 


Please use the sendbug(1) utility to report bugs in the system. 
Before reporting a bug, please try to reproduce it with the latest 
version of the code. With bug reports, please try to ensure that 
enough information to reproduce the problem is enclosed, and if a 
known fix for it exists, include that as well. 


# 


Figure 3. OpenBSD 5.7-BETA, close to the next release 


From a company point of view, if a new feature is gen- 
uinely needed and if it is not possible to do it internally, 
sponsorship might be considered an option. 


Bug acceptability level 

Indeed, the -CURRENT branches introduce potentially some 
new bugs. In the case of FreeBSD, for example, recently the 
Random Number Generator framework change, which was 
made pluggable, was found to be broken. Instead of coming 
back to the previous version, which sounds less risky, the 
issue was fixed — | personally prefer this kind of approach. 
On my side, | run FreeBSD with some local fixes (for bsdgrep, 
for example), some were merged upstream, hopefully some 
others will be in the near future. 
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DMAXUSERS=89 -D_KERMEL -MD MP -e ../../../../devsipeLsit_eye.e 


ec =Werror -Wall -Wimplicit=function-declaration = Wno=sain -wno=-uninitialized -Wrrame=larger=thane2e4? -semodelekernel =sno0-red=zone =nn0=55e2 =-sno=-55e =ano-Jdnow 
—tha-bui ltin-wenprintt -—Tno-bulltin-loeg -fha-builtin—-log? -fro-builtin-malloc -foo-pie -O2 -pipe -nostdine -Toufaakaa fas 


r —fno-builtin-printt -foo-builtin-snprintt 


=and—sax —esott=-float -fno-onit-frame-pointe 
-T, -Teafeafeefss ffPeh -DDOS -DOLAGHOSTIC —-D 


KTRACE -BACCOUNTING -DFMEMSTATS -DPTRACE -DRARNL_DERLIA: -DCRYPTO -DSYSVMSG -DS7SVS5H -DS7SVSHM SVE _SaAP ENCRYPT -DCOMPAT_43 -OFFS -DFFS2 -OFFS_SOFTUPDATES -DUFS_OTRHASH -DQU0TA -IMEXTZFS -ORFS -DMFSCLIENT -DNFSSE 


RVER -DCD966@ -DUDF -DMSDOSFS -DFIFO -DTMPFS -DFUSE =050CKET_SPLICE -OTCP_SACK -DTCP_ECN -DTCP_ SIGNATURE -DINET -DINETG —DIPSEC “DPPP_ESOCOMP = 


PPP_DEFLATE -DPIPEX -DMRQUTING -OMPLS -DRQ0T_CONFIG -DUSER_PCICONF 


=DAPERTURE -DATRR -ONTFS —0DHIBERNATE PPC VERERIGE —MISEVERBOSE -OWSDISPLAT_OOMPAT_USL -DWSDISPLAY_COMPAT_FAWKEO -DSDISPLAY_DEFAULTSOREENS="6" —DWS0TSPLAY_COMPAT_PCWT —DXB6GEMU -DONEWDREVERBOSE —DRULTIPROCESSOR — 


DMAXUSERS=i@ -0_KERMEL -4D -HF =c .. fdew/pei/draysradeoni/radeon_gart.c 
ec -Werror -Wall -Wimplicit- 


r -fno-builtin-printt -fro-builtin-snprintt -fno-builtin-wenprint? =-fno-builtin-log 


funetion- se rat tion -Wag—aain -Wre-uninitialized 6 -Wirame—larger—thane=2647 -nomodelekerne] -ano-red-rone -ine-s9e2 -Eno-39e “ann Sdnow 
=fno-builtin-log? -fro-builtin-malloac -foo-ple -02 -pipe -nostdine -T../../../.. -I. =I. 


hit irs att flopt -fne-onit-frame—pointe 
farch -DD08 -DOTAGNOSTIC -0 


of 
KTRACE -DACCOUNTING —DKMEMSTATS -OPTAACE -DPL_DERIMG -DCRYPTO -DS7SVMS6 -OS 75584 -057SV5SHM —2V_SWAP ENCRYPT -OCOMPAT_43 -DFFS -DFFS2 -OFFS_SOFTUPDATES ~DUFS.| DIRMASH erat DEXTZFS -DHFS -DNFSCLIENT -ONFSSE 
RAVER -DCOSbb@ -DUDF -DRSDR0SFS -DFIFO -DTAPFS -OFUSE -050CKET_SPLICE -01CP_ SACK -DTCP_ECN -DTCP_SIGHATURE -DINET -DINET6 -DIPSEL -OPPP_LBSOCORP -DPPP_DEFLATE -DPIPEX “aw “UTING -DHIPLS -DRR0T_ CONFIG —-DUSER_PLICONF 


~DAPERTURE -ONTRR -ONTFS -DHIBERNATE -DPCIVERBOSE -DUSEVERBOSE -DWSDISPLAY_COMPAT_USL -OWSDISPLAY_COMPAT_RAWKEO -DWSDISPLAY_DEFAULTSCREENS="6" —DWSDISPLAY_COMPAT_PC| 


DMAMUSERS=f8 “BL KERMEL mi HP -¢ of. 

Sh .. fected. . fCont/newvers. sh 

ec aaaertie “wall 
r =fno-builtin- =printt =fno-bui Ltin- snprintt 


sfoofe SOS pelfdray radeai/ raga. ¢ 


fno-builtin-vsnorintt -fne- bul Ltin= log 


—Wimpli rit funetion-declaration -Wno-sain -wro-uninitialized -Wft ram Lorger—thanecea? —fonadé lskernne] -ano-red-Tone -no-s5e02 -—no-$5e@ —Eno- ae 
‘fmo-builtin-Loge -fno-builtin-malloc -fno-pie -f2 aera =nostd Ticfaass =I, =Tiafeafsaf 


“DXEGEMU -DONEWDREVERBOSE -DAULTIPRECESSOR - 


—anid—aax -esott-Tlost -foro-onit-Trame—pointe 
farch “0008 -DDLAGMOSTIC -2 


TRACE “DACCOUNTING -DKHEMSTATS -DPTRACE -DPOOL,_ DEBUG -CCRYPTO -DSYSVHSG —DS7SVSBH “DSYSVS SHA —DUVE_SwAP_ENCRYPT -DCOMPAT_43 -DFFS -DFFS2 -DFFS_ SOFTUPDATES “DUFS OTRHASH =OQU0TA ~DEXTIFS “GHFS -OMFSCLIENT -ONFSSE 
RAVER -DCDSb58 -DUDF -DMSD0SFS -DFIFO -DTMPFS -DFUSE -2S0CKET_SPLICE -OTCP_SACK -OTCP_ECN -DTCP_SIGNATURE -DINET -OINET6 -DIPSEC -DPPP_ESOCOMP -—OPPP_DEFLATE -DPIPEX -DMROUTING -—0MPLS -OR00T_COMFIG -DUSER_PCICOMF 


=DAPERTURE =O87TRR =0NTFS <DHIBERMATE =DPCIVERBOSE =DUSEVERBOGE =OWSDISPLAY_OOMPAT_USL =<OWS0TSPLAY_COMPAT 


DMAXUSERS=f8 -O_KERNEL -4D -HP -¢ Wers.¢ 


_RAWKED =O0S01SPLAY_DEFAULTSOREENS="6" 


=—(WS01SPLAY_COMPAT_PCVT =(X86EMU =20NEWDIREVERBOSE =(MULTIPROCESSOR = 


sofoofee fen farch/faméa/contfid.script —K —warn-comsen =<nopie —$ =~ <0 bed $(575TER_HEAD) vers.o 4{0615) 


hex data bss dec hex 
BE29S97 HMEI6S SORE1G GEFbRG Gtabdc 
onp =5 bed fbsd || in -f /bsd sobsd 


ev /nbs id /bsd 
g 


Figure 4. Recompiling OpenBSD is a quite simple task 


DragonFly Oragonflame 4.1-DEVELOPMENT DragonFly v4.1.8.816.9898T3e-DEVELOPMENT #4: Sat Feb 21 12:42:62 GMT 2615 


rooted ragont lame: fusrfobj/usr/src/sys/X86_64 GENERIC x86_64 


Dragontlame# git diff —cached > ~/patch-sysctlL.diff 


Figure 5. DragonflyBSD uses git, better for branches handling 


In the case of OpenBSD, the new XHCI driver 
(for USB 3.0) still does not work completely. For example, 
recently a memory leak was found in dhclient (but fixed) ... 
But nothing really major, OpenBSD -CURRENT is runnable 
daily as well. 


DragonflyBSD had memory leaks in the kernel and in 
hammer filesystem. Once again, they were fixed promptly. 


The bug “acceptability” level depends on whether you're 
willing to patiently take the time to make explicit bug re- 
ports in case the bug in question is blocking, or fixing them 
internally and pushing those fixes upstream. But there is 
no support to expect — again a point to consider well. 


Contribution 

Most of the contributions are done in the -CURRENT 
branches. That makes perfect sense as the -CURRENT 
branches are the perfect areas for both fixes and innova- 
tive features, adding disruptive changes whereas the re- 
leases/stables welcome the fixes only. It also makes more 
sense for -CURRENT that recompiling the system is the 
natural usage. 


lf you are a quite advanced BSD user and you wish 
to contribute to make them better for the whole community 
then using the development branches can be considered. 
There are many areas, not only purely technical (like the 
documentation) which can be improved. 


DragonflyBSD uses git internally and due to its branch- 


ing model, it is pretty handy to create a proper diff to sub- 
mit it for review. 
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Conclusion 

Most companies stick to stable/release versions with only 
security fixes. If your applications rely on specific API/ABI 
versions, it is indeed better to keep on doing it. 


Somehow, few others run experimental branches. 
Indeed. For example, Yahoo uses FreeBSD -CURRENT 
internally for their servers. 


Given the short life release cycle chosen by Open- 
BSD with its fair amount of disruptive changes (ie., every 
6 months), it is less surprising to find users using the de- 
velopment branch. 


| recompile quite often FreeBSD / OpenBSD base sys- 
tems but for those who have no interest at all in doing it, 
some snapshots builds are made fairly often ... 


Saying that, it is advised to be registered in the relevant 
mailing lists: freebsd-current@freebsd.org, tech@open- 
bsd.org, tech@netbsd.org, commits@dragonflybsd.org. 


David Carlier has been working as a software developer since 2001. 


He used FreeBSD for more than 10 years and starting from this year, 
he became involved with the HardenedBSD project and performed 
serious developments on FreeBSD. He worked for a mobile product 
company that provides C++ APIs for two years in Ireland. From this, 
he became completely inspired to develop on FreeBSD. 
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The goal of this tutorial is to walk readers through 

the installation and the basic configuration of Postfix, 

one of the most popular SMTP servers, and SoamAssassin, 
which will be used for basic e-mail filtering. 


es to and between servers. The delivery of e-mail 

to user-facing software, such as e-mail clients like 
Thunderbird, Outlook and others, is the job of other proto- 
cols like IMAP (and the old, obsolete POP3). 

SMTP was made for a smaller and more trustworthy In- 
ternet and offers next to no guarantees that a message 
was sent from a valid user, to a valid user, or that it will 
arrive in time. However, it offers decent micro-guarantees 
about what happens if a message is received by a serv- 
er. In particular, it offers store-and-forward semantics in 
which the SMTP server receiving a message promises 
that, if it acknowledges that the message was received, it 
has successfully stored the message and will do its best to 
forward it to its intended recipient. The first feature makes 
sending unrequested and forged e-mail very easy (we call 
it soam), and the second feature makes processing e-mail 
fairly resource intensive, as it involved synchronous writ- 
ing of the messages on the server's drives. Because of 
this, running e-mail servers is harder than it should be. To- 
day, SMTP servers are heavily guarded by firewalls, anti- 
viruses and strict rules about who is permitted to send e- 
mail through them. 


S MTP is the protocol used to route e-mail messag- 


SMTP and DNS 

A modern e-mail message has two parts: the username 
and the domain part. We will cover the username part lat- 
er, but the domain part needs some special consideration. 
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As when accessing a web site, an application needs 
to use the DNS system to find out the IP address of the 
server which is accessed. To allow e-mail to be served by 
different servers than those which serve other services 
for the domain, a special type of DNS entry is necessary, 
called the “MX record” (stands for Mail eXchanger). When 
sending e-mail to an address such as “user@example. 
com’, first the MX record for “example.com” is searched. 
If it’s not found, a regular “A” record is searched. 

Note that DNS must be resolvable for e-mail to work, 
as is true with other services, such as the web. 

In addition to MX records, the DNS system can also car- 
ry SPF records (Sender Policy Framework) which can be 
used to inform the world about which IP addresses are al- 
lowed to send e-mail on behalf of which domains. This can 
be used to reduce the possibility of forged e-mails for cer- 
tain domains. 


Complex e-mail routing 

In some cases, the system which receives e-mail for a do- 
main is not the final server which will store the e-mail in the 
users mailboxes. These cases require that e-mail be rout- 
ed from a server to a server, usually from a more general 
server (such as a global organization’s server) to a more 
specific server (Such as a local branch's server). In such 
a scenario, it is possible for the servers which are inter- 
nal to the organization to have local IP addresses, though 
it requires a careful design of the network and its services. 


02/2015 


E-mail usernames 
On Unix-like systems, the usernames used in e-mail ad- 
dresses are usually the system usernames. In such sys- 
tems, e-mail is delivered to locations provided by the 
system, such as the users’ home directories, and are pro- 
tected by the system access protection rules (such as file 
ownership and access permissions). 

This is not necessarily so: e-mail usernames could be 
stored in a database and delivered to special mailboxes, 
but such setups are outside the scope of this article. 


Spam protection 

Since incoming e-mail messages to an e-mail server are 
unauthenticated and can be easily forged, their content 
needs to be analysed and classified in addition to simple 
checks such as the “To” and “From” addresses. Modern 
anti-spam tools use heuristics and actually process the 
content of the message. Since different people receive 
different types of messages, the best of such systems 
adapt their heuristics to personalize them for each user. 


Disabling Sendmail 

FreeBSD is shipped with Sendmail as the default e-mail 
server system. Sendmail is enough for very simple usage, 
but quickly gets very complicated when additional features 
need to be configured. Since the goal of this tutorial is to in- 
stall Postfix, Sendmail needs to be disabled before Postfix 
can function by adding the following line to /etc/rc. conf: 


sendmail enable=”NONE” 


A reboot is recommended to stop Sendmail listening on 
various network ports. 


Installing and configuring Postfix 
Postfix can be installed from a package with a command 
such as: 


# pkg install postfix 


When asked about activating Postfix in the /etc/mail/ 
mailer.conf, answer ae 

Postfix configuration files are located in /usr/local/ 
etc/postfix. 

lts main configuration file is main.cf, and this is the file 
which will be modified in the next steps. 

Postfix is very careful about which e-mail to receive, 
and the first line of defense is specifying the domain for 
which it will act as an SMTP end-point server. By default, 
this domain will be extracted from the server's host name, 
specified in the myhostname directive, such as: 
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myhostname = mail.example.com 


The next step is to configure the domains which will be 
accepted as “local” for mail delivery: 


mydestination = Smyhostname, localhost.Smydomain, 


localhost, Smydomain 


A common configuration is for the SMTP server to be 
configured to accept e-mail from clients in the same lo- 
cal network without authentication. This is convenient 
for the users as it skips the requirement for SMTP login, 
but can escalate into a problem if one of the local ma- 
chines gets taken over by malware which will (ab) use 
the server for sending spam e-mail. 

To configure Postfix to accept connections from IP 
addresses in the same subnets as the server as “trust- 
ed”, configure mynetworks style: mynetworks_ style = 
subnet. 

This method will automatically detect the server’s IP 
addresses and subnets. In case manual configuration 
is required, use the mynetworks directive. Some setups 
(for example, servers behind ADSL connections) require 
that e-mail is not routed directly (which is the default be- 
haviour) but is always relayed by the ISP’s e-mail server. 
In this case (and only in this case), use the relayhost di- 
rective to specify the “upstream” e-mail server: 


relayhost = mail.myisp.com 


Such setups usually require that the connection to the 
upstream server is authenticated with the username and 
password provided by the ISP. This can be achieved by 
adding the following lines to the main.c¢ file: 


smtp sasl auth enable = yes 
smtp _sasl_ password maps = hash:/usr/local/etc/postfix/ 
sasl passwd 


smtp sasl security options = 


The password file specified in the 7 
password maps needs to be created with the following 
content: 


smtp. 6asl 


mail.myisp.com username:password 


This file can contain multiple lines, each specifying 
a server's name and the username and password used 
when connecting to it. Usually, only one line is required. 
Since this is a security sensitive file, you should adjust 
its file access permissions as needed. 
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This text file needs to be converted into Postfix’s “hash- 
map” format by issuing the following command: 


# postmap hash:/usr/local/etc/postfix/sasl passwd 


For historical reasons, the /etc/aliases file needs to be 
parsed and a hashmap file created by issuing the 
newaliases Command: 


# newaliases 


Lastly, Postfix needs to be enabled in /etc/rc.conf 
by adding the following line: 


postfix enable = “YES” 


It can be immediately started by issuing a command 
such as: 


# service postfix start 


Sending an example e-mail message to test Postfix 

The built-in FreeBSD program named “mail” can be used 
to send an example e-mail message. You can specify the 
e-mail Subject value with the “-s” argument, and when the 
program starts, it will read a message directly from the 
console. You should write an example message and end 
it with Ctrl-D: 


> mail -s “A test message” ivoras@example.com 
An example message here. 
[Cir lp. 


The Ctrl-D keyboard combination will create an “end-of- 
file” signal to the reading program. To check if the e-mail 
was successfully delivered, check the /var/log/maillog 
file and check that a file was created for the user given in 
the above command in the /var/mai1 directory. 


Installing SoamAssassin 

SpamAssassin is a framework for very configurable and 
adaptable e-mail analysis. It has multiple optional plugins 
which enhance its core functionality. SpamAssassin can 
be installed with the following command: 


# pkg install spamassassin 


Its configuration files are located in /usr/local/etc/mail/ 


Spamassassin. 
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After the installation, you should run the sa-update utility 
to refresh the database of common spam patterns. You can 
do this by adding a line like the following into /etc/crontab: 


* * * * | root /usr/local/bin/sa-update 


Enable the SpamAssassin daemon by adding the follow- 
ing line to /etc/re.conf: 


spamd_ enable="YES” 
Start the daemon with a command such as: 
# service sa-spamd start 


Integrating SpamAssassin with Postfix 

Asmall helper shell script is required to integrate SoamAs- 
sassin with Postfix. For this for this article, we will name 
It /root/spamfilter.sh and give it the following content: 


#!/bin/sh 
SENDMAIL=/usr/local/sbin/sendmail 
SPAMASSASSIN=/usr/local/bin/spamc 

# logger <<<“Spam filter piping 

to SpamAssassin, then to: SSENDMAIL $@” 
S{SPAMASSASSIN} | S{SENDMAIL} “S@” 


exit $? 


Don’t forget to make the file executable. 
Next, modify the /usr/local/etc/postfix/master.cf 
file. The first “smtp” line needs to be changed to: 


smtp 
inet. fk -— - — = smtpd =—o 


content filter=spamfilter 
In addition to this, one new line needs to be added: 


spamfilter 

Unis th fi = 

— pipe 

flags=Rq user=spamd 


argv=/root/spamfilter.sh -oi -f ${sender} ${recipient} 
After these modifications, Postfix needs to be restarted: 
# service postfix restart 

Testing SpamAssassin 


The configuration described in master.cf enables spam 
filtering on the “smtp” service which is bound to the TCP 
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port 25. This means that e-mail sent directly through 
Unix local delivery will not be filtered and SpamAssas- 
sin cannot be tested with the “mail” command. To test 
SpamAssassin, you need to send e-mail through the 
SMTP port 25. 

There is a special string which can be used for test- 
ing. If this string is found in the message by SpamAssas- 
sin, the message will receive 1000 spam points and be 
marked as spam. This string is: 


XJS*C4JDBQADN1 .NSBN3* 2 TDNEN* GTUBE-STANDARD-ANTI-UBE-TEST- 
EMATL*C.34X 


Installing and configuring Dovecot as an IMAP 
server 
IMAP is a protocol for e-mail message retrieval, used by 
user-facing applications to fetch messages and offer them 
in some sort of user interface. In contrast to the old POP3 
protocol, IMAP offers a unified “view” of the message da- 
tabase on the server (messages are usually not deleted 
from the server when retrieved), supports subdirectories 
of the main mailbox and offers some rudimentary ability 
to share mailbox folders between different users. 
Dovecot is one of the most popular IMAP server appli- 
cations. It is well-written and extensible, and cooperates 
well with Postfix. 


How e-mail is delivered 

An SMTP server (operating in one of the roles called a Mail 
Transfer Agent or Mail Submission Agent) accepts a mes- 
sage and then either relays it to another server or attempts 
to deliver it into a “local” mailbox (meaning a mailbox on 
the server where the SMTP server is running). During the 
process of the delivery, the message may be processed 
in a number of ways, for example, by scanning it for spam 
(as seen in the previous tutorial). The delivery is performed 
by a module of the SMTP server called a “delivery agent.” 
A “mailbox” is usually a single text file to which all the e-mail 
is concatenated, together with some special lines which 
delimit it. An alternative standard is called “maildir’ which 
stores each message in its own file, in a special directory 
structure. Once the message is safely written to this stor- 
age, the SMTP server’s job is done. 

Another type of server, for example, the IMAP server, 
implements a protocol by which an application for reading 
e-mail retrieves and presents these messages to the user 
(such applications are called Mail User Agents). The IMAP 
server needs to find the messages and is configured com- 
pletely separately from the SMTP server. In addition to sim- 
ply presenting the messages from the mailbox file through 
the IMAP protocol, Dovecot can perform some interesting 
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additional features, especially when integrated with Post- 
fix. By default, Dovecot will maintain an indexed database 
of messages, which greatly speeds up all operations with 
them, but also contains a specialised delivery agent which 
is smarter than a plain SMTP delivery agent and can filter 
certain types of messages into certain IMAP folders. 


Installing Dovecot 

The current version of Dovecot is available in the pack- 
age named dovecot2, and the Sieve message filtering 
module is in the package named dovecot-pigeonhole. 
Those packages should be installed with the usual pkg 
command: 


# pkg install dovecot2 dovecot-pigeonhole 


lts configuration files’ directory iS /usr/local/etc/ 
dovecot, but the directory is empty after the installation 
and needs to be populated first. The directory: 


/usr/local/share/doc/dovecot/example-config 


contains example configuration files, all of which should 
be copied into Dovecot’s configuration directory (pre- 
serving the directory structure, i.e. the conf.a subdirec- 
tory). In the same way, copy the files from /usr/local/ 
share/doc/dovecot-pigeonhole/example-config/conf.d into 
Dovecot’s conf.d directory. 

For a basic configuration, you should modify the follow- 
ing files, and make sure that the specific configuration 
lines are present and uncommented in them: 


dovecot.conf 
Enable only IMAP with the following line: 


protocols = imap 


conf.d/10-auth.conf 


Disable non-encrypted plaintext logins, include the de- 
fault system authentication mechanism, and specify the 
correct plugins directory: 


disable plaintext auth = yes 
'include auth-system.conf.ext 
mail plugin dir = /usr/local/lib/dovecot 


conf.d/10-mail.conf 
Specify that the default user mailbox is found in /var/mail, 


but that the additional mailbox files for IMAP folders will 
be in the directory ~/maii for each user separately: 
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mail location = mbox:~/mail: INBOX=/var/mail/%u 


You should also modify the locking methods configuration 
to skip the “dotlock” method which is tricky to use secure- 
ly when the default mailbox is located in /var/mail: 


mbox read locks = fcntl 
mbox write locks = fcntl 


conf.d/10-ssl.conf 


Specify where the TLS certificates are and which cipher 
suite to use (the same ones used for Apache): 


ssl_ cert =</var/ssl/ivoras.net.crt 

ssl_key =</var/ssl/ivoras.net.key 

Ss Cipter list = 

'ADH: !|EXPORT: !SSLv2:EECDHt+aRSA+AESGCM: EBECDH+aRSA+RC4:RC4+R 
5SA:+HIGH:+MEDIUM: +LOW 

eont.d/1lo-lda cont 


Specify that the local delivery agent will use the Sieve 
plugin: 


protocol lda { 
mail plugins = $Smail plugins sieve 
} 


conf.d/15-mailboxes.conf 


In this file, modify all of the mailbox sections and add 
a line: 


auto = subscribe 
to each uncommented section. 
conf.d/90-sieve.conf 


Enable some convenient Sieve extensions: for modifying 
the e-mail headers and for manipulating IMAP message 
flags: 


sieve extensions = t+editheader timap4flags 


Using the Dovecot local delivery agent in Postfix 

In order to make use of the Sieve filtering plugin, Postfix 
needs to be configured to pass the e-mail which would be 
delivered locally to Dovecot’s local delivery agent module. 
This is done very simply, with the following line in Postfix’s 
main.cf file: 


mailbox command = /usr/local/libexec/dovecot/dovecot-lda 
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-f “SSENDER” -a 
“SRECIPIENT” 


Restarting Postfix and Dovecot 
Before using Dovecot, enable it in /etc/rc.conf with a line 
such as the following: 


dovecot. enable="ihs" 


Postfix and Dovecot can be restarted by issuing the fol- 
lowing commands: 


# service postfix restart 


# service dovecot restart 


Creating Sieve rules 
Sieve rules can be created globally, for all users, or with 
user-specific scripts. This tutorial will describe the per-us- 
er scenario (for the global case you should look at the 
sieve default directive in 90-sieve.conf). By default, 
Dovecot-Pigeonhole will try to find a Sieve script file 
named ~/.dovecot.sieve in each user’s home directory. 

Sieve has a programming language which is designed 
for simple rule-based e-mail processing. It is a power- 
ful language which can perform many actions, but the 
most common uses of Sieve are for filtering soam mes- 
sages and for sorting e-mail messages into separate 
IMAP folders. 

An example Sieve script can be as follows: 


require [“fileinto”, “envelope”, “imap4flags”, “regex”, 
“editheader”, variables” ]; 

if header :contains “X-Spam-Flag””YES” { 

fileinto “Junk”; 

} elsif address :contains “to””ivoras@example.com” { 

addheader “Importance””High”; 

addflag “Slabel3"; 

} elsif address :is “to””mailing-list@example.com” { 

fileinto “mailing-list”; 

} else { 

keep; 

} 


The above script will perform the following actions on 
each message: 


1. If the e-mail headers contain the SpamAssassin’s 
flag, save the message into the “Junk” folder 

2. If the message's “To” header mentions my address 
directly, mark the message as important (this is so 
that messages which contain my address only in the 
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CC field or which are passed through mailing lists are 
marked as “less important”) 

3. If the message is sent to a specific mailing list, save it 
to a separate folder 


Sieve can do much more, and you should study the ex- 
amples given at goo.gl/QGBgDS. 

You can test that everything is working by sending some 
e-mail which will match the above Sieve rules (after re- 
starting Postfix and Dovecot). It is easy to make syntax er- 
rors in Sieve, but luckily the Pigeonhole Sieve module will 
log such errors into a file named ~/.dovecot.sieve.log. 

Be sure to check /var/log/maillog for error messages! 


Connecting to the IMAP server 

When configuring an e-mail reading application, 
you should connect to the IMAP server on the standard 
port 143, and use TLS for secure network traffic, which 
includes logins. 


Installing and configuring the RoundCube 
webmail application 

RoundCube is a “normal” PHP application which offers 
the user the ability to login with a username and password 
to an IMAP server. It collects the messages and folder 
from the IMAP server and displays them in a nice graphi- 
cal interface. It also uses a database for storing miscella- 
neous information such as user preferences and the con- 
tacts list (address book), but the amount of information 
stored to the database is very small (it does NOT store 
e-mail messages in the database; the messages are only 
stored on the IMAP server). 


Installing and configuring RoundCube 
RoundCube requires the following PHP module to be in- 
stalled in addition to those installed for ownCloud: 


php5-filter 

Similarly to how ownCloud was installed in Tutorial #6, this 

tutorial will explain how to install RoundCube from current 

official sources, put into the /srv/www/roundcube directory. 
As a first step, download the source archive from http:// 

roundcube.net/download/ and put it into /srv/www: 


# cd /srv/www 

# fetch 

http://sourceforge.net/projects/roundcubemail/files/ 
roundcubemail/1.0.3/roundcubemail-1.0.3.tar.gz/download 

# tar xzf download 

# mv roundcubemail-1.0.3 roundcube 


# chown -R ivoras roundcube 
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The application needs to write logs and temporary files, 
so the appropriate paths should be allowed to be written 
by the web server: 


# chgrp -R www roundcube/temp roundcube/logs 


# chmod -R gtrw roundcube/temp roundcube/logs 


RoundCube needs to be configured by copying the 
config.inc.php.sample into CONG imc. pnp in the config 
subdirectory, and modifying the following configuration 
variables: 
Sconfig[ ‘db dsnw’] = ‘mysql://roundcube@localhost/ 
roundcube’ ; 

Sconfig[ ‘des key’] = ‘Imka9f84mrandomrandom123'; 
sconfig[ ‘mime types’] = ‘/usr/local/etc/apache24/mime.types’ ; 
Sconfig[ ‘default host’] = ‘tls://localhost’; 


Sconfig[ ‘preview pane’] = true; 


g 
Sconfig[ ‘preview pane mark read’] = 2; 

Ssconfig[ ‘enable installer’] = true; 

The first line configures the database configuration. 
For it to be valid, you should create the roundcube 
database in MySQL and grant the roundcube user all 
rights on it: 


# mysql 


Welcome to the MySQL monitor. Commands end with 
; Or \g. 

Your MySQL connection id is 1 

Server version: 5.5.40 Source distribution 

Copyright (c) 2000, 2014, Oracle and/or its affiliates. 
All rights reserved. 

Oracle is a registered trademark of Oracle Corpora- 
tion and/or its affiliates. Other names may be trademarks 
of their respective owners. 

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current 
input statement. 


mysql> create database roundcube; 
(0.00 sec) 


mysql> grant all on roundcube.* to 


Query OK, 1 row affected 
’roundcube’ @’ localhost’; 


Query OK, 0 rows affected (0.02 sec) 

The second line in the configuration file needs to speci- 
fy a unique key used by RoundCube to securely transmit 
some session-related information. The third line enables 
the built-in installation process, and will need to be re- 


moved before RoundCube is used in production. 
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Next, Apache’s virtual host configuration for the TSL 
host on port 443 needs to be modified to allow access to 
the newly installed PHP application: 


Alias /mail “/srv/www/roundcube” 
<Directory “/srv/www/roundcube”> 
Options ExecCGI FollowSymLinks 
AddHandler fcgid-script php 
FCGIWrapper /usr/local/bin/php-cgi 
eae) 

DirectoryIndex index.php 
AllowOverride All 

Require all granted 


</Directory> 


The reason why it is necessary to allow access to Round- 
Cube only from a SSL-enabled virtual host is because, like 
ownCloud, it requires a login through the web page, so its 
username and password need to be protected. 

RoundCube also requires some configuration changes 
to PHP. By default, there is no php. ini in a freshly installed 
PHP on FreeBSD, but there are two example files named 
php.ini-development And php.ini-production IN /usr/ 
local/etc. You should copy the production version of the 
file into the pnp. ini file and change the following lines: 


date.timezone = Europe/Zagreb 
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Next, RoundCube’s internal installation process needs to 
be started by visiting the /installer path in its installa- 
tion, such aS https://example.com/mail/installer. If ev- 
erything is OK, you should arrive at a button to initialize 
the database, which you should click. 


Conclusions 

After database tables are created, you can optionally run 
the tests for the SMTP and IMAP servers (the default con- 
figuration assumes that they are both on localhost), then 
remove the installer line from config.php and the install- 
er directory from RoundCube sources, then visit its main 
URL at https:/example.com/malil. 


Ivan Voras is a FreeBSD developer and a long-time user, starting with 
FreeBSD 4.3 and throughout all the versions since. In real life he is a 
researcher, system administrator and a developer, as opportunity 
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ing to cloud computing. He is currently employed at the University of 
Zagreb Faculty of Electrical Engineering and Eomputing and lives in 
Zagreb, Croatia. You can follow him on his blog in English at http:// 
ivoras.net/blog or in Croatian at http://hrblog.ivoras.net/, as well as 
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BigData 


TECHCON 


April 26-28, 2015 


seaport World Trade Center Hotel 
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Big Data TechCon is the HOW-TO technical conference 
for professionals implementing Big Data solutions 
at their company 


Come to Big Data TechCon to learn the best ways to: 
e Process and analyze the real-time data pouring into your organization 


Learn how to extract better data analytics and predictive analysis 
to produce the kind of actionable information and reports your 
organization needs. 


Come up to speed on the latest Big Data technologies like Yarn, Hadoop, 
Apache Spark and Cascading 


Understand HOW to leverage Big Data to help your organization today 


Big Data TechCon” is a trademark of BZ Media LLC. A Event 
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The Basics of The GDB 
Debugger 


To be able to inspect a program more easily, we need to 
have the symbol table available for the program we intend 
to debug; this is accomplished by using the -g flag of 

the compiler we are going to use (we could also debug it 
without the -g flag but it is really cumbersome sometimes). 
In our case we will use FreeBSD 10 as the platform and the 


clang compiler that comes with it. 


fter a program is compiled using the —g flag we 
Ae able to peek inside it using the gdb debugger. 

To start a debugging session. All you need to type 
is the following: 


# gdb <program name> 


And we will see a (gdb) prompt. That means that we are 
ready to start typing gdb commands (Figure 1). 

Or if the program we need to debug is currently running, 
we must type: 


#qgdb 
#(gdb) attach <pid of running program> 


7 


leiraatr ‘s 
WL gab b. 1.1 | Bo 
opyright 2004 Free Sof 
1B is free software, covered by 


and/or 


UeOS.- 


is. I LE ee ee | () 


rree 


tware 


Foundation, 
the GNU 


for GDB. 


amd 


Figure 1. GDB example 
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Lenera 
distribute copies of 


the conditions 


Let's start with some basic commands and _ inspect 
a running application. Dor this example | have selected 
this application http://freeciv.wikia.com/wiki/Main_ Page. 

“Freeciv is a Free and Open Source empire-building 
strategy game inspired by the history of human civili- 
zation. The game commences in prehistory and your 
mission is to lead your tribe from the Stone Age to the 
Space Age...” 

We will inspect the game structures at runtime with gdb. 
Let's follow these steps: 


e¢ Edit /etc/make.conf and add the line witH DEBUG=yes 
(this will not strip your binaries so you will have the 
symbol table and also add the debug flags to the 
compiler when compiling the sources of your ports) 


— 
Erie = 


marce|-Treebs 


02/2015 


Game Government View Orders Reports Editor Help 


rh 


roPpulstion: @ 
Year: 4000 BCE (TO: 
Gold: 50 (*@) 

Tax? @0 Luw: O Seis 60 
(Click For more info> 


UFO fe JG fo fe fo fe fe 
P|.) .|'S |e 


Explorer 


Hover: i 


Plains (BufFalo> 


to the Freeciv version 2.4.3 Server 
leg@ged if af ‘créira' conmected to Creira, 


shed cormtrol over the server, 


players are ready: starting game, 


pl m OOD /USr; fDinys Treeciv-server 
NU gdb 6.1.1 [Freesbsp) 

opyright 2004 Fr 
iB 15 free software, covered by the GNU General Public License, 
elcome to change it and/or distribute copies of 


ee Software Foundatlion, Inc. 


ype “show copying” to see the conditions. 


heré 15 absolutely no warranty Tor GDB. 
his GDB was configured as “amd64-marcel-Treebsd"... 


gdb) Fr 
itarting program: 
LWP 100405] 


usr/ local /bin/freeciv-server 


Thread 805406400 (LWP 100405/Treeciv-server)] 
1s the server for Freeciv version 2.4.3 
fou can learn a lot about Freeciv at http: //wew. freeciv.org, 


freeciv-server program has 


Loading rulesets. 


AI*1 has been 
AI*? has been 
AI“3 has been 
AT*4 has been 
Al“5 nas Deen 
Now accepting 


Wy 


added as Easy level Al-controlled player 


added as Easy level Al-contro! led 
added as Easy level Al-control led 
added as Easy lével Al-contro| led 


added as Easy level Al-controlled player 


new ClVvent connections 


help, type ‘help’. 


Figure 3. The GNU General Public License 
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it under 


Type “Show warranty” 


—e 
i] 


You have command sccess 


Gu 


level 


and you are 
certain conditions, 


detar!s. 


player authentication support, but 1t°s currently not in us 
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rogram received signal SIGINT, Interrupt. 
Switching to Thread 805406400 (LWP 100405/Treeciv-server) | 
©) from /lib/libc.so.7 
gdb) bt 
O Ox0000000801dd606a in Select CC 
PO801a87b02 in select () from /1ib/libthr.so.3 
Mooddsoodcd203 in Fc_select (ne-?, readtds-Ox7Tttifitdobs, writefds-Ox/tTTttttfd638, 


) from /lib/libc.so.7 


OxOO0O0008008Ta58a in server_sniff_all_input ¢) at sernet.c:686 
OxO00000080090F407 In Srv_running () at srv_main.c:2317 

OxOO0000080090e0a4 in srv_main () at srv_main.c:2//? 

Ox00000000004026ca In main (argc=1, argqv=O0x7TTttttfdasas) at civserver.c:453 


Figure 4. Jo continue the execution 


¢ Install freeciv from ports Now we will use our first gdb command: 
¢ Start the freeciv server and client (freeciv-server and 

freeciv-gtk2) # gdb /usr/local/bin/freeciv-server 
¢ Join your local game (Figure 2) 
As we don't know anything about how Freeciv works, 
we will press CTRL-C. This will interrupt the program 
and we will take it from there. For starters, let’s interrupt 
and see where we are. If we want to continue the execu- 
tion, we type ‘continue’ or ‘c’ (Figure 4). 

Figure 5 is a screenshot from the client program freeciv- 
gtk2; we need to join our local game as we are going to 
debug the server (Figure 5). 

The #<num> you see are the stackframes of simply called 
frames. When your program is started, the stack has only 
one frame, that of the function main. This is called the initial 
ea frame or the outermost frame. Each time a function is called, 
zs eee a new frame is made. Each time a function returns, the frame 


— wersion 2.4.3, guiegtk-2.0 client 


‘rogram received signal SIGINT, Interrupt. 
OxOO00000801ddb06a In select ¢) from /11b/11bc.50.7 
(gdb) bt 
‘O Ox0000000801dd606a in select () from /lib/libc.so.? 
OxOOOO000801a8/bO? In select €) from /lib/libthr.so.3 
OxOOoOoONs00dcd703 in fc_select (ne?7, readfds-Ox7tftttttfd6es, writefds-Ox/tttttfTd66s, 
exceptfds-Ox7fffffffdse8, timeout-Ox7ffttfffd5ds) at netintf.c:126 
() at serne 

S0T407 In srv_running () at Srv_mMain.c:231/7 

190e0a4 In srv_main () at srv_main.c:2/7?? 
0D04026Ca In main (argc=l, argv=Ox/Ttttttfdass) at civserver.c:453 
( 
‘4 =OxO00000080090T407 in srv_running () at srv_main.c:2317 
2317 while (server_sniff_all_input() -- S_E_OTHERWISE) { 
Current language: auto; currently minimal 


L 
J 


log_debug("snifftingpackets"); 
check_Tor_full_turn_done(); /* HACK: don't wait during AI phases 
while (server_sniff_all_input() =-- 5_E_OTHERWISE) { 

nothing *, 


L 
J 


“ After sniff, re-zero the timer: (read-out above on next loop) * 


(qdb) 


Figure 5. The client program freecivgtk2 
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conn_list_iterate(game.all connections, pconn) f{ 
if (srvarg.auth_ enabled 
&& !pconn->server.15_closing 
£& pconn->server.status != AS ESTABLISHED) { 
auth process status(pconn): 


} 


} conn_list_iterate_end 


if (5_5 RUNNING == server_state() && game.info.timeout == 
(void) send_server_info_to_metaserver(META_REFRESH): 
return S_E_ENO_OF_TURN_TIMEQUT; 


if (fc_select(max_desc + 1, &readfs, &ewritefs, &exceptfs, atv) 


(void) send _ server_info to metaserver(META_ REFRESH); 
if (game.info.timeout > 
oo 5 5S RUNNING == server_state( ) 
$a game.server.phase_timer 
£& (read timer _seconds(game.server.phase timer) 
> gamé.info.seconds to phasedone)) { 
con prompt off({); 
return oe 


Pf 


log_debug("sniffingpackets"): 
check_Tor_Tull_turn_done(); /* HACK: don't wait during AI phases */ 
while (server_sniftf_all_input¢) == S_E_OTHERWISE) j 

™ nothing “/ 


After sniff, re-zero the timer: (read-out above on next loop) */ 


if (fc_select(max_dedc + 1, &readfs, swritefs, &exceptfs, &tv) <= 0) { 


| "a 
ew 


Pago! Nagle Fass 


LA ae Lad A 


log_debug¢"sniffingpackets”");: 
check_for_Ttull_turn_done(); /* HACK: don't walt during AI phases 
while ¢server_sniff_all_input() == 5_E_ OTHERWISE) 

™ nothing 


| +h 


fr 


bad had hed ha Lad 


Ne ee 


im 


— ‘ 


After sniff, re-zero the timer: (read-out above on next loop) 


r J 


J 


il 


a58a in server_sniff_all_input () at sernet.c:686 
select(max_desc +1, &readts, dwritets, d&exceptts, &tv) == 0) { 
: 
Breakpoint 1 at Ox8O008Tabl1/7: Tile sernet.c, line 695. 
(adb) §j 


Figure 7. The innermost frame 
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for that function invocation is eliminated. If a function is re- 
cursive, there can be many frames for the same function. 
The frame for the function in which execution is actually oc- 
curring is called the innermost frame. This is the most recent- 
ly created of all the stack frames that still exist. Let's go into 
frame 3; to do this we type either ‘frame 3’ or ‘f 3’ (Figure 6). 

It seems that the server is going to send us end of turn. 
Let’s make sure to set a break point, the format 


ie 
<break|b> <source.c>:<line number> 


(gdb) b sernet.c:695 


‘gdb) fT 3 
F360 | OXNO000008008fa58a In server_sniff_all_input 


0) 
if (Tc_select(max_desc +1, &readts, & 


last_noplayers = 0 

ronnections = Talse 

i = 5 

jax_desc = 6 

sxcepting = false 

eadTs = Tds_bits = {89, 0 <repeats 15 times=}} 

ritefs { fds_bits = <repeats 16 times> | 
{1 _Tds_bits = {8&, 15 to 


Tv_usec 


mes> ty 


lt seems we are wrong, let’s interrupt again and inspect 
the data at this point (Figure 8). 

Typing ‘i lo’ means info locals which will display all lo- 
cal variables in this frame and their values, which is pret- 
ty handy. Let’s take a look at something easier to see. 
Sometimes in freeciv, another civilization will try to negoti- 
ate terms with us. Looking at the source code, we find the 
add_ clause function in the diptreaty.c source code. That 
function will add a term which will make the other part ac- 
cept or reject our terms (Figure 9 and Figure 10). 

After playing a few minutes we hit this break point. 
At this point, we don’t even know which civilization has 


dt Sernet.c:686 
writetTs, 


ane Ki i a [ } l f ‘ - RT VV ) 


»/freeciv-2.4.3/common 


Add clause to treaty. 
bool add_clause(struct Treaty “ptreaty, 
enum Clause_type type, int val) 

L 

Struct player “*pto = (ptrom — ptreaty-=plrod 
? ptreaty--pirl : 

struct Clause *pclause: 

enum diplstate_type ds 


= Player_diplstate_get(ptreaty--plro, ptreaty 


cype CLAUSE LAST J 


og_errarc” 
return FALSE; 


16 times>}} 
D <repeats 15 times>}} 
{tv_sec = 1, tv_usec 
(gdb) b add_clause 
akpoint 1 at Oxs600cT9436: 


file diptreaty.c, line 138. 


T 


_ontinuing 


ireakpoint 1, 


Figure 10. The add_clause function 
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Lllegal clause type encountered.”) 


add clause (ptreaty-0x806db9eed0, ptrog-0x8067dd400, 
at diptreaty.c:138 
fodb) § 


Struct player 


ptreaty->pilrd): 


-pirl) 


> Type ; 


Line: 138 PC: Ox800cT9436 


type-CLAUSE_CEASEFIRE, val—0) 
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approached us to negotiate terms. Now we can know | assume the the negotiation civilization should be in the 
ahead of time as we set the breakpoint where the nego- _— pfrom pointer (Figure12). 
tiation starts (Figure 11). 


Add cl alse to t ty. 


bool ‘uct Treaty “ptreaty, 
enum clause type type, 


“oto 


t t Clause 
enum “dip! state_ | 
playe _dip]s sta te_get(ptreaty--plro, ptreaty->plrl)->type: 


| i (1 ¥PE -, i i oe “i: fc | ALISE_ | AST) 
lag_errorc"Tllegal clause type 


eebsd- th Thread 60540604 In: add_ eran Line: 138 PC: Ox800cTS436 
Li "}, SkI)1_level = AT LEVEL EASY, fuzzy = 300, expand - 10, science_cost - 100, warmth - Q, 
frost Oo, barbarian_type NOT _A BARBARIAN, love (1 <repeats 175 times>}}, a OxBO1OS91bO, 
was_Created false, 15_connected true, current_conn = 0x0, connections = Ox80687c170, 
gives_shared_vision {VEC 0" repeats 15 times>}, wonders {0 <repeats 200 times=}, 
attribute_block = {data Oxo, length . 0 » attribute_block_buffer «= idata = © ‘0, length = 0}, 
tile_knows nm = ibits 3888, ¥ec 3400 ""}, rob = Ox8064b97c0, {server = {Status = {vec 
got_Tirst_city false, Sate any = Ox806400000. really_gives_vision = { 
5 Times>;, debug = {vec = ;, adv = Oxg05 430800, als = (Ox806820000, Oxo, 
, Gelegate_To = “0° <repeats 47 Times:, orig_usernamée = ‘0° <repeats 47 Timess};, client = ; 
1 WISTON = ({1b1TS = 1, Vec = OxX806400000 “+, {bits O, Vec = OXOF FTE 


Figure 11. /n the pfrom pointer 


gdb) p ptfrom 
10 = (struct player *) 0x8063dd400 
qdb) § 


Figure 12. /n the pfrom pointer 


a , ay ’ 
struct player_slot *slot; 
char name[48]; 
char username[48]: 
char ranked_username[48]: 
Int user_turns: 
_Bool 15_male: 
struct government “government; 
Struct government *target_government; 
struct nation_type *nation; 
--Type <return> to continue, or q <return> to quit---f 


Figure 13. /n the pfrom pointer 


1= 0, Trost = 0, barda rian Sie Le He A_BARBAR Dah ove = 10, U, U, U, QO, 
repeats 123 Tihes>}}, Wal S created - false, 15 connected - Talse, 
current_conn Ox0, connects ons KO Be 360" gives_shared_vision [Wet "0" <repeats 
wonders = {0 <repeats 21 times:, 129, 0 <repeats 178 times>}, attribute_block = {data = 0: 
length = OF}, SEPbate Diack MAkice . {data - 0x0, length - 0}, tile_known - {bits - 3888, 


WeC = OXBO5643800 “"}, Pob = OxX8064dalea0, (Server = {Status Wee =  ‘'000°})}, gottirst_city 


private map Oxe0e52c000, real lyugives_vision [VeC "0" <repeats 15 times>}, debug 
VEC = aly adv = Ox805 agen, ais = {0x80626T000, Oxd, reiaiatal 

delegate_ O°" <repéa 47 times>, orig_username = ‘\0° 47 times>}, client = { 

Cile_vision = {ibIts 257. vec - Ox80652C000 ~ >, (oOItS = O, vec = OXOTTTT| 


gdb) p “pfromil 


Figure 14. /n the pfrom pointer 
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To print the variable’s values, we just type ‘p’. In this case 
‘p is a pointer to a player structure. If we can check the defi- 
nition of the player structure, we just type ptype pfrom and 
the structure definition will be displayed (Figure 13). 

Now let's see what the values are for these fields for the 
demanding civilization. As the pfrom is a pointer we need 
to use pointer notation to check its contents (Figure 14). 


20, 
c 


population = 
Oo, units_buil 


Lif 


t 


And there we go the full dump for the player struct (Fig- 
ure 15). Looking at the player struct, it seems that the lead- 
er name is Roy Jenkins and looking at the backtrace (bt), 
the clause of the treaty seems to be “cease fire”, So we are 
going to be offered a peace treaty (Figure 16). To continue 
executing the program type ‘next’ or ‘n’; something like this 
will be displayed in the diplomacy tab (Figure 17). 


EPL LM. 

OX80684/0cC8, Team 
| tt 
ru 


= Ovk0540c 


5 
aT 


t, 


iplomacy.c: 90° 
;OOROORce 


Figure 16. A peace treaty 


Game Edit View Selact nit work Combat Civilization Help 


| lem pars - European 
[+i] Master Jacques de Molay ay 
| | Gold: “dh Add Clause... 
nko Europeans 
Eales at (T3) Chief Roy Jenkins 
Gold: 9 = ab Add Clause... 


Figure 17. /n the diplomacy tab 
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freebsd- th Thread 8054064 
id 0x000000080098af 73 in 


In: add_clause 

Ganage_auto_explorer (punit-O0x8054d?7 
do_explore rappel at 
Srv_running () at Srv vega riz6] 
srv_@ain () at srv_m 

Gain (argc-l, argv-Ox7 


Game saved as freeciv-TO009-¥-3550-auto.sav.bz2 
Ereakpoint 2, add_clause (ptreaty=0x8064b9ee0, 


at diptreaty.c: 13: i 


? ptreaty- 
struct Clause “pclause-: 
enum diplstate_type ds 
player_diplstate_get(ptreaty 


‘piri 


>plro, ptreaty 
y | p =e CLAUSE_LAST) { 
log_error(’ 11 legal clause type encountered.") 
return FALSE; 


ADVANCE Sa 24 
tech 


Jalid_ advance 
Value %1 in 


(type CLAUSE 
log_error¢€’ 11 legal 


return FALSE: 


(15_ secs clause(type) 
DS_PEACE && type = A 
DS_ARMISEICE do Typ CLAUSE 
OS_ALLIAKCE && type CLAUSE 
0S CEASEFIRE && type CLAUSE 


(gdb) Tf O 
#0 add_clause (ptreaty=0x8064b9ee0, 
(gdb) 115t 


pfrom=(x80630d400, 


1151 
Cype 
LAUSE EMBASSY 
oe CLAUSE EMBASSY 
= @num clause_type {CLAUSE 
LLAUSE CESSEFIRE, CLAUSE 


(gdb) § 


AD L Br a 
AD VA, | — fi Li 


WSE_ GOLD, CLAUSE MAP, 
PEACE, CLAUSE iTA 


ALLIANCE, CLAUSE 


6900) 
unittools.c 


Line: 143 
at alitoexplorer.c: 396 


-f44/ 


CivSserver.c: 


prrom=0x8063dd400, 


ptreaty->plrd); 


>pirl) 


PEALE.) 
ALL LANCE 


CEASEFIRE) }} 


Type=(LAUSE_EMBASSY , val 


CLA 
VISION, 


type=CLAUSE_EMBAS 


USE_SEAMAP, 
LAUSE_EMBAS 


name(nat jon_of_player(ptre 
name (nation_of_plar 


rule 
rule 


Ni a i 1 ‘on 
nation 
FALSE; 


recurn 


if (type ==j{ CLAUSE_EMBASSY && | 
* we already have emb 
iog_errar( ITllé@gal embassy ¢ 
nation_rule_ 

nation_rule 


F Ai ch 
oo ee ee 


WSsy 

ause: BS alt 
name (nation_oft_player(pto)), 
name (nation_ot 
return 


if €'qame.info.trading_gold && 
return FALSE; 


type == CLAUSE_GOLD) 


tech && CLAUSE_ 


if ('game. 


Info. trading 
FALSE: 


type -- 
‘eTUPh 


it (loame into. trading_city && type =-- CLAUSE CITY) 


Figure 20. The next 
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r(ptreaty 


player_ has_real_embassy(pto, 
pady have embassy 


player(ptrom))): 


1 


ADVANCE | } 


Vale) 


‘Type; 


by number (val) f 
clause Me : Wal }: 


) 


Lin@: 143 4PCt: 


CLAUSE CITY, 
¥, CLAUSE_LAST} 


=} at dy ptreaty.c:1: 


GDB Debugger 


PC: Ox800cT’ 


Oxsoocto4a4 


a t “yW—=f) ] ro y), " 
>pir1)) 


ofrom)) { 
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EW OTD Ome y slivglusir - Tale req (Pplayersvaslos0ocugd, COUNTE 
values ri at diplhand.c 

OxOOo000080088c84 2 in server. handle_packet (type=}PACKET_DIPLOMACY_CREATE_CLAUSE REQ, 
packeT={0x80680d0750, pplayer=-{0x8063dbc00, pconn-O0x800c37580) at hand_gen.c:2773 
OxOOCOOO0EO090cCb1b In Server_packet_input (pconn-0x800c77580, packet-0x8068007250, type-99) 
at 5rv_main.c:1622 I 

OxOOOMO08008TbESD in incoming client_packets (pconn=0x800c37580) at sernet.c:460 
OxOO00M008008Taabe in server_sniff_all_input () ia sernehet; oan 
OxOOCO00080090T407 In srv_running () at srv_main 

OxO00000080090e084 In Srv_main () at henge aga 


rin fsa Thala y _ hoe es 5, — : S aTrirr 


Figure 21. The commands 


dip inand.c : 
player “number (pother) , giver, type, 
value); 

disend_packet_diplomacy_create_clause(pother-=connections, 
player_number(pplayer), giver, type, 
a 


eaty EeaTAC pat lier: TSS Streatyys 


“TT eeoeeeteeeeee een ee. 


Cancel meeting. No sanity checking of input parameters, 50 don't call 
this with Input directly froe untruste ed source 
a. Hee aes ee ret TT Tet rT Terria. ee eR Be ea en a oe en ee. | 
static void cual ty diplomacy_cance | mecting(struct player “pplayer, 
struct player “pother) 


struct Treaty “ptreaty =- find_treaty(pplayer, pother): 
if (ptreaty) 


an ‘th Thread 8054064 In: handle “do lomacy. “create c “lause. Pet Line: 692 PC: Oxaerds3a? 
--Type <return> to continue, or g <return to quit--- 

F Ox0D0O00000000K4076Ca 1h Main (aragce1l, afov-Ox7TTTTTTTda at civserver.c:453 

gdb) n 

gab ) 

gdb } 

db } 

gdb ) 


"FIXME: should this be put in a ruleset somewhere? 
#deTine TURNS_LEFT 16 


py 
> 


fa =j OA 


ee Pertti det bht cet bt eee a a a ae eo Ptihbbht eee ae ae 


Calls treaty_evaluate function if such is set for AI player. 
fbbbbbtbbbbbtbbdbbbtbbbbbt bb bbbtbbbbbbtbbdbbbtbbababbb baba bbe bbb eb adhd eae ah be eae ae : 
static void call_treaty_evaluate(struct player *pplayer, struct player *aplayer., 

Struct Treaty “ptreaty) 


if (pplayer->ai_controlled) { 
CALL_PLR_AI_FUNC(treaty_evaluate, pplayer, pplayer, aplayer, ptr 


7 oe lL Re 


Ss fo =] 1 


i es 
a LE: 


gels le ote oul ls eRe lr al: o> leer le all a ae eee te op area ctl > abe eee ell ope oa eae ae ele lr omy eae oie ele le oat one ee ele oe onl melee le nl one oiler oc cea oh 


Calls treaty_accepted function if such 15 set for AI player 


Ce ee ee ee 


Static void call_treaty_accepted(struct player “*pplayer, struct player *aplayer, 
struct Treaty “ptreaty) 


FC: oxg0087d314 


mb 
gdb ) 
gdb) 
gdb) 
gdb ) 
gdb} 
gdb) 11st 
gdb) step 
li_treaty_evaluate (pplayer-0x18063dbc00, aplayer-0x8063dd400, ptreaty-0x8064b9e9e0) at diplhand. 


Figure 22. The commands 
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What you cannot see in the screenshot is that | have re- Let's go line by line using next; you could also use the 
quested an embassy in return for the cease-fire treaty, but step command but if you use the step command, it will 
here, it is shown on Figure 18. take you inside a function call instead of just evaluating 


sidered. it a question aout money 
Vold rites icing orteeenaeiaes player aaneimaes struct iaiee ‘avaae 
Struct Tréaty “ptreaty) 

1 

Int total_balance = OQ: 

bool only_giTts = TRUE: 

enum cdipls state _type ds_atfter - E 

player_dip|lstate_getipplayer, aplayer)-> type: 
int given_cities = @: 


Clause_1l1St_iterate(ptreaty--clauses, pclause) | 
a I 


if (1S_pact_c lause(pclause->type)) { 
ds_after = pact_clause_to_diplstate_type(pclause->type): 


(pc lause->type -- CLAUSE CITY && pclause->from -- pplayer) f 
Given_citiese+: 
; 


Clause_list_iterate_end: 


roy Tray at ees 693 
db) list 
db) 5 
ll_treaty_evaluate (pplayer=-0x%806300d400, aplaye ptreaty- at diplhand.c:? 
db) list 
db) 5 
db) list 
db) 5 
i_treaty_evaluate (pplayer=0x8063dd400, aplayer=-0x8063 . Ptreaty=0x8064b9ee0) at advdiplomacy.c:578 
db) list 
db) 


Figure 23. The following commands 
aovelp homacy .c 


if (given_cities > 0 
&& city_list_size(pplayer-=cities) - given_cities <= 7) 
always keep at least two cities 
DIPLO_LOGC(LOG_DIPL2, pplayer, aplayer, “Cannot give last cities"): 
return; 


‘ Accept if balance 15 good * 
if (total_balance == 0) { 
handle_diplomacy_accept_treaty_regq(pplayer, player_number(aplayer)); 
DIPLO_LOG(LOG_DIPL2, pplayer, aplayer, “balance was good: Xd" 
total_balance); 
} else { 
AT complains about the treaty which was proposed, 
made the proposal. */ 
if (pplayer !'= ptreaty--plro) { 
notify(aplayer, _("*%s (AT)* This deal was no 
player_name(pplayer), 
Jlayer_name(aplayer)); 


freebsd-th Thread 8054064 In: dai_treaty_evaluate Line: 671 PC: Ox80094ac1 
1: total_balance = 0 

€odb) 11st 

(odb) b advdipjomacy.c:621 

Breakpoint 3 at Ox#0094acl13: file advdiplomacy.c, line 621. 

(gab) c 

Continuing. 


Breakpoint 3, dai_treaty_evaluate (pplayer-0xs063dd400, aplayer-Oxs063dbc00, ptreaty-0x8064b9ee00) 
at advdiplomacy.c:671 
L: total_ halance — -450 


Figure 24. The commands for program execution 
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the function and returning like the next command 
(Figure 19). 

We are currently at line 143; | just checked what kind of 
data type was CLAUSE EMBASSY, it was a enum one 
(somewhat obvious). Using next a couple of times will get 
us to the next step. See Figure 20. 

Keep on typing ‘n’ and we will exit from the function call and 
arrive to handle diplomacy create clause req (Figure 21 ). 

Let's keep on typing ‘next’ and we will arrive to this func- 
tion call treaty evaluate. That seems interesting. May- 
be the results of rejection or acceptance of conditions are 
done. As | explained earlier, we can step into this one us- 
ing the step command (Figure 22). 

Let's step all the way to get to another point in the program 
execution; after a couple of steps are shown on Figure 23. 

So a quick glance at the source code tells us that the 
total_ balance variable is somewhat important to evaluate 
if a clause is accepted (In our case we are requesting to 
give us an embassy). Instead of printing this variable mul- 
tiple times, let’s leave it available in the display. 


#(gdb) display total balance 


Then we set a breakpoint somewhere ahead of advd 
iplomacy.c:621, WE Can SEE that the total _ balance value Is 
displayed and it is -450 (seems bad for our proposal). 

AS we Can Séé@, total balance >=0 Is the condition to 
approve the proposal. This is a review of the commands 
used in this session: 


O Print values and names of all 
local variables in the current 
scope. 


info local Nl 


backtrace bt A backtrare is a summary of 
how your program got where it 
is. It shows one line per fatale, 
for many frames, starting with 
te currently executing frame 
(frame zero), followed by its 


caller (frame one), end on up 


the stock. 
frame f <frame The call stack is divided up 
<frame number> into contiguous pieces called 
number> stack frames, or frames for 


short; each frame is the data 
associated with one call to one 
function. The frame contains 
the arguments. 
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given to the function, the 
function ‘s local variables, 
and address at which the 
function is executing. 


print p <variable> displays the value of the 
<variable> variable 
display disp <variable> Will automatically print the 
<variable> value the variable being 
displayed as long as it is within 
the scope 
win Win Will enter gdb in tui (text user 
interface) mode if we did 
not entered in the first place. 
Default layout is source at the 
top commands at the bottom. 
next n Execute next line of code. Will 
n <number not enter functions. You can 
of next to use as parameter the number 
perform> or times to execute next 
step S Step to next line of code. 
s<number Will step into a function. 
of step to 
Perform> 


These are really basic commands, but really useful. 


Advanced inspection of data structures 
and variables 
Now that we have used the display command or the print 
command, it is getting pretty tedious to manually inspect 
a variable or data structure by typing p or display every 
time we hit a breakpoint we have set. There is a command 
called commands to save us from all this typing. 

First we set a breakpoint where we want to automatically 
inspect data. In this case I'll check one of the city functions. 


(odi5) B-caity.e22352 
(gdb) 4 breakpoint keep y 0x0000000800cflb7b in citizen_ 
base mood at 


Cley.c@r2 352 
Now we can type the following: commands <breakpoint number> 
commands 4 


(gdb) 


Type commands for when breakpoint 4 is hit, one per line. 
End with a line saying just “end”. 


After you have set the instructions to be executed after 
the breakpoint is hit, you could modify them or just erase 
them like this: 
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(gdb) commands 4 


Type commands for when breakpoint 4 is hit, one per line. 
End with a line saying just “end”. 


> end 

Now if you want to execute something: 

(gdb) commands 4 

Type commands for when breakpoint 4 is hit, one per line. 
End with a line saying just “end”.> printf “Setting city 


mood for leader: %s”, pplayer->name 


> end 


Now we can type all the instructions we want to be execut- 
ed when this breakpoint is hit. Usually, we use print to dis- 
play values, but there is a more powerful function called 
printf that uses a similar format as the C-language function: 
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(gdb) printf “Ss”, pplayer->name 
As in C printf, ordinary characters in the template are 
printed verbatim, while conversion specification intro- 
duced by the ‘% character causes subsequent expres- 
sions to be evaluated, their values converted and format- 
ted according to type and style information encoded in 
the conversion specifications, and then printed. 

For example, you can print two values in hex like this: 
printf “foo, bar-foo = 0x%x, 0x%x\n”, foo, bar-foo 
printf supports all the standard C conversion specifica- 
tions, including the flags and modifiers between the ‘%’ 
character and the conversion letter, with the following 
exceptions: 

The argument-ordering modifiers, such as ‘2$’, are not 
supported. 

The modifier ° 
or width. 


* is not supported for specifying precision 


MIY Cd SUGEENIe@ Smt i SECO lS ON AT) national go AAS AE WSIS a . IVIES os 
aigalvatias airlines, ere PR i ae alicls services) ae freelance cantina fA my current role as a ern equaliser 


| have successfully advocated and implemented Open Source solutions in an organisation previously moribund by proprietary and 
closed source products, generating substantial cost savings as a result. 


The most effective teams are built on strong, communicative, interactive relationships. 


Paradoxically, | combine a “hard” technological mindset with strong “soft” people skills. A Myers-Briggs type INBjJ, it is essential for me 
that any culture | serve in embraces integrity, creativity, humanity and ethics. Ergo, quality is not something you believe in, quality 


is something you experience. 


My IT career and professional philosophy has been shaped by the late Michael Kidron, author of “The State Of The World Atlas’, and 
Béla Hatvany, founder of Silver Platter Information Ltd, who | am both deeply indebted to for hiring me in my first IT role. If you are 


looking for a truly different type of IT guru, please get in touch. 


linuxgreybeard@gmail.com 
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The *” flag (for separation of digits into groups according 
to LC_NUMERIC’) is not supported. 

The type modifiers ‘hh’, ‘j’, ‘t’, and ‘z’ are not supported. 

The conversion letter ‘n’ (as in “%n’) is not supported. 

The conversion letters ‘a’ and ‘A are not supported. 

Note that the ‘Il’ type modifier is supported only if the un- 
derlying C implementation used to build GDB supports the 
long long int type, and the ‘L’ type modifier is supported 
only if long double type is available. 

As in C, printf supports simple backslash-escape se- 
quences, such as \n, ‘\t’, ‘\V, ‘V”, ‘\a’, and ‘\f, that consist of 
backslash followed by a single character. Octal and hexa- 
decimal escape sequences are not supported. 

Additionally, printf supports conversion specifications 
for DFP (Decimal Floating Point) types using the follow- 
ing length modifiers together with a floating point specifier. 
Letters: ‘H’ for printing Decimal32 types. 

‘D’ for printing Decimal64 types. ‘DD’ for printing Deci- 
mal128 types. 

lf the underlying C implementation used to build GDB 
has support for the three length modifiers for DFP types, 
other modifiers such as width and precision will also be 
available for GDB to use. 

In case there is no such C support, no additional modi- 
fiers will be available and the value will be printed in the 
standard way. Here’s an example of printing DFP types 
using the above conversion letters: 


printf “D32: SHf - D64: SDf - D128: SDDF\ 
nj l«2345df,1.2b10dd,1.2b1d1 


Dynamically allocated arrays 
Sometimes. it’s better to put most of the type we will 
need to take a look at in the contents of dynamically 
allocated arrays (the ones created by malloc and calloc 
system calls). 

For example we have the usual static memory array: 
Char ¢/ 8001). 
It's easy to display its contents using 
(gdb) pt 
But what about this one: 
Tit: SOP an 
t = (int *) malloc ( 8001 * sizeof( int) ); 


(gdb) pt 


This will give only the address 
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(gdb) p *t 


This will give you the data of the first element in the ar- 
ray, so what is the solution? 


(gdb) p *t@25 


This command will print 25 elements from the array t; the 
format is pointer@<number of elements. 


Getting information from the symbol table 
When we compiled our program with the —g flag, we in- 
structed the compiler to generate a symbol table in our 
program binary. This table contains variable names, func- 
tion names and types. Now let’s suppose we want to know 
the names of all the functions available. We could use one 
of the info family commands: 

(gdb) info functions 

This command will print the names and data types of 
all defined functions. If we want to check only the func- 
tion names matching a regexp we use the command: in- 
fo functions <regexp>. 


For example: 


(gdb) info functions city 
Will match all functions that have city string in their 
name, you must use grep regexp not perl’s regexp. 
The same goes with variables with the command: 
(gdb) info variables 
Print the names and data types of all variables that are 
declared outside of functions (not the local variables). 
Also the same syntax for info variables regexp (gdb) in- 
fo variables city. Print the names and data types of all vari- 
ables (except for local variables) whose names contain 
a match for regular expression regexp. 
(gdb) info address symbol 
Describe where the data for the symbol is_ stored. 
For a register variable, this says which register it is kept in. 
For a non-register local variable, this prints the stack-frame 
offset at which the variable is always stored. Note the con- 
trast with ‘print &symbol’, which does not work at all for 
a register variable, and for a stack local variable prints the 
exact address of the current instantiation of the variable. 
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(gdb) whatis exp 


Print the data type of expression exp. exp is not actual- 
ly evaluated, and any side-effecting operations (Such as 
assignments or function calls) inside it do not take place. 
Any kind of constant, variable or operator defined by the 
programming language you are using is valid in an ex- 
pression in GDB. 


(gdb) whatis 
Print the data type of $, the last value in the value history. 
(gdb) ptype typename 


Print a description of data type typename. typename 
may be the name of a type, or for C code it may have the 
form ‘class class-name’, ‘struct struct-tag’, ‘union union- 
tag’ or ‘enum enum-tag. 


(gdb) ptype exp 
ptype 


Print a description of the type of expression exp. ptype 
differs from whatis by printing a detailed description, in- 
stead of just the name of the type. For example, for this 
variable declaration: 


struct example {double dtype; float ftype} exl; 
The two commands give this output: 


(gdb) whatis exl 

type = struct example 
(gdb) ptype exl 

type = struct example { 
double dtype; 

float ftype; 

i 


As with what is, using ptype without an argument refers 
to the type of $, the last value in the value history. 

(gdb) info types regexp 

Print a brief description of all types whose name 
matches regexp (or all types in your program, if you 
supply no argument). Each complete typename is 
matched as though it were a complete line; thus, 


‘| type value’ gives information on all types in your 
program whose name includes the string value, but 
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‘i type “value$’ gives information only on types whose 
complete name Is value. 

This command differs from ptype in two ways: first, like 
whatis, it does not print a detailed description; second, it 
lists all source files where a type is defined. 

(gdb) info source 

Show the name of the current source file—that is, the 
source file for the function containing the current point 
of execution—and the language it was written in. (gdb) 
info sources. 

Print the names of all source files in your program for 
which there is debugging information, organized into two 
lists: files whose symbols have already been read, and 
files whose symbols will be read when needed. 


(gdb) Into Tunetions 


Print the names and data types of all defined functions. 


(gdb) info functions regexp 

Print the names and data types of all defined functions 
whose names contain a match for regular expression regexp. 
Thus, ‘info fun step’ finds all functions whose names include 
step; ‘info fun “step’ finds those whose names start with step. 
(gdb) info variables 

Print the names and data types of all variables that are de- 
clared outside of functions (i.e., excluding local variables). 
(gdb) info variables regexp 

Print the names and data types of all variables (except 


for local variables) whose names contain a match for 
regular expression regexp. 


Conclusions 

In GDB we have three ways of interrupting the program 
flow and inspecting what we need; breakpoints, watch- 
points and catchpoints. 

A breakpoint stops the execution at a particular location 
within the program. We have temporary breakpoints, reg- 
exp breakpoints and we could set conditional breakpoints. 

The usual breakpoint : 


(gdb) break <source>:<line> 


(gdb) break <source.c>:<function> 


(gdb) break 3 This one stops at line 3 of the current 
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source file being executed. 


(gdb) break <function> 


The temporary break point is a simple breakpoint that is 
deleted after it is hit; the command for this is: 


(gdb) tbreak <same format as breakpoint> 


The regexp breakpoint sets breakpoints at the functions 
matching the regexp provided 

(gdb) rbreak “cityConditional breakpoint, stops the 
execution of the program only if the condition is met 


(gdb) b if stremp (commands [0].synopsis,”*start”) == 


Yes, you could use the C library functions as long as 
your program is linked against libc. 

You can enable or disable breakpoints with the follow- 
ing command: 


enable once — Enable breakpoints for one hit 

enable delete — Enable breakpoints and delete when hit 
(gdb) enable once 1 

(gdb) enable delete 1 


A watchpoint stops the execution when a particular 
memory location (or an expression involving one or more 
locations) changes value. Depending on your system, 
watchpoints may be implemented in software or hard- 
ware. GDB does software watchpointing by single-step- 
ping your program and testing the variable’s value each 
time, which is hundreds of times slower than normal exe- 
cution, but it’s really useful if you really don’t have a clue 
of where the problem is in your program. 
The syntax for this command is: watch <expr> 


(gdb) watch commands [0] 


Watchpoint 1: commands [0] 


A catchpoint stops the execution when a particular event 
occurs. The event could be one of the following. 
Raised signals may be caught: 


Gatch signal — all signals 


catch signal <signame> - a particular signal 
Raised exceptions may be caught: 
¢ catch throw — all exceptions, when thrown 


¢ catch throw <exceptname> — a particular exception, 
when thrown 
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¢ catch catch — all exceptions, when caught 
¢ catch catch <exceptname> — a particular exception, 
when caught 


Thread or process events may be caught: 


¢ catch thread_ start — any threads, just after creation 
¢ catch thread_exit — any threads, just before expiration 
¢ catch thread_join — any threads, just after joins 


Process events may be caught: 


¢ catch start — any processes, just after creation 

¢ catch exit — any processes, just before expiration 
¢ catch fork — calls to fork() 

¢ catch vfork — calls to vfork() 

¢ catch exec — calls to exec() 


Dynamically-linked library events may be caught: 


¢ catch load — loads of any library 

¢ catch load <libname> — loads of a particular library 

¢ catch unload — unloads of any library 

¢ catch unload <libname> — unloads of a particular library 


The act of your program’s execution stopping may also 
be caught: 


¢ catch stop 

¢ C++ exceptions may be caught: 

¢ catch throw — all exceptions, when thrown 
¢ catch catch — all exceptions, when caught 


You can enable and delete breakpoints, watchpoints and 
catchpoints with the enable and delete command. 


Carlos Neira has worked several years asa C/C++ developer and 
kernel porting and debugging enterprise legacy applications. He is 
currently employed as aC developer under Z/OS, debugging and 
troubleshooting legacy applications for a global financial company. 
Also he is engaged in independent research on affective computing. 
In his free time he contributes to the PC-BSD project and enjoys met- 
al detecting. 
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A Complete 
Guide to FreeNAS 


Hardware Design, 


Part Il: Purpose and Best Practices 


past overdue by now. For that, we apologize. The issue was the depth and complexity of 

the subject, as you'll see by the extensive nature of this four part guide, due to the variety 
of ways FreeNAS can be utilized. There is no “one-size-fits-all” hardware recipe. Instead, there 
is a wealth of hardware available, with various levels of compatibility with FreeNAS, and there are 
many things to take into account beyond the basic components, from use case and application 
to performance, reliability, redundancy, capacity, budget, need for support, etc. This document 
draws on years of experience with FreeNAS, ZFS, and the OS that lives underneath FreeNAS, 
FreeBSD. Its purpose is to give guidance on intelligently selecting hardware for use with the 
FreeNAS storage operating system, taking the complexity of its myriad uses into account, as well 
as providing some insight into both pathological and optimal configurations for ZFS and FreeNAS. 


f\ guide to selecting and building FreeNAS hardware, written by the FreeNAS Team, is long 


A word about software defined storage 

FreeNAS is an implementation of Software Defined Storage; although software and hardware are 
both required to create a functional system, they are decoupled from one another. We develop and 
provide the software and leave the hardware selection to the user. Implied in this model is the fact 
that there are a lot of moving pieces in a storage device (figuratively, not literally). Although these 
parts are all supposed to work together, the reality is that all parts have firmware, many devices re- 
quire drivers, and the potential for there to be subtle (or gross) incompatibilities is always present. 


Best Practices 

ECC RAM or Not? 

This is probably the most contested issue surrounding ZFS (the filesystem that FreeNAS uses to store 
your data) today. I’ve run ZFS with ECC RAM and I've run it without. I’ve been involved in the FreeN- 
AS community for many years and have seen people argue that ECC is required and others argue 
that it is a pointless waste of money. ZFS does something no other filesystem you'll have available to 
you does: it checksums your data, and it checksums the metadata used by ZFS, and it checksums 
the checksums. If your data is corrupted in memory before it is written, ZFS will happily write (and 
checksum) the corrupted data. Additionally, ZFS has no pre-mount consistency checker or tool that 
can repair filesystem damage. This is very nice when dealing with large storage arrays as a 64TB 
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pool can be mounted in seconds, even after a bad shutdown. However if a non-ECC 
memory module goes haywire, it can cause irreparable damage to your ZFS pool that can 
cause complete loss of the storage. For this reason, | highly recommend the use of ECC RAM with 
“mission-critical” ZFS. Systems with ECC RAM will correct single bit errors on the fly, and will halt the 
system before they can do any damage to the array if multiple bit errors are detected. If it’s imperative 
that your ZFS based system must always be available, ECC RAM is a requirement. If it's only some 
level of annoying (slightly, moderately...) that you need to restore your ZFS system from backups, 
non-ECC RAM will fit the bill. 


® 


How Much RAM is needed? 

FreeNAS requires 8 GB of RAM for the base configuration. If you are using plugins and/or jails, 
12GB is a better starting point. There’s a lot of advice about how RAM hungry ZFS is, how it requires 
massive amounts of RAM, an oft quoted number is 1GB RAM per TB of storage. The reality is, it’s 
complicated. ZFS does require a base level of RAM to be stable, and the amount of RAM it needs to 
be stable does grow with the size of the storage. 8GB of RAM will get you through the 24TB range. 
Beyond that 16GB is a safer minimum, and once you get past 100TB of storage, 32GB is recom- 
mended. However, that’s just to satisfy the stability side of things. ZFS performance lives and dies 
by its caching. There are no good guidelines for how much cache a given storage size with a given 
number of simultaneous users will need. You can have a 21TB array with 3 users that needs 1GB 
of cache, and a 5007TB array with 50 users that need 8GB of cache. Neither of those scenarios are 
likely, but they are possible. The optimal cache size for an array tends to increase with the size of 
the array, but outside of that guidance, the only thing we can recommend is to measure and observe 
as you go. FreeNAS includes tools in the GUI and the command line to see cache utilization. If your 
cache hit ratio is below 90%, you will see performance improvements by adding cache to the sys- 
tem in the form of RAM or SSD L2ARC (dedicated read cache devices in the pool). 


RAID vs. Host Bus Adapters (HBAs) 

ZFS wants direct control of the underlying storage that it is putting your data on. Nothing will make 
ZFS more unstable than something manipulating bits underneath ZFS. Therefore, connecting 
your drives to an HBA or directly to the ports on the motherboard is preferable to using a RAID 
controller; fortunately, HBAs are cheaper than RAID controllers to boot! If you must use a RAID 
controller, disable all write caching on it and disable all consistency checks. If the RAID controller 
has a passthrough or JBOD mode, use it. RAID controllers will complicate disk replacement and 
improperly configuring them can jeopardize the integrity of your volume (Using the write cache 
on a RAID controller is an almost sure-fire way to cause data loss with ZFS, to the tune of losing 
the entire pool). 


Virtualization vs. Bare Metal 

FreeBSD (the underlying OS of FreeNAS) is not the best virtualization guest: it lacks some virtio 
drivers, it lacks some OS features that make it a better behaved guest, and most importantly, it 
lacks full support from some virtualization vendors. In addition, ZFS wants direct access to your 
storage hardware. Many virtualization solutions only support hardware RAID locally (I’m looking 
at you, VMware) thus leading to enabling a worst case scenario of passing through a virtual disk 
on a datastore backed by a hardware RAID controller to a VM running FreeNAS. This puts two 
layers b ZFS and your data, one for the Host Virtualization’s filesystem on the datastore 
and ano the RAID controller. If you can do PCI passthrough of an HBA to a FreeNAS VM, 
and get all the moving pieces to work properly, you can successfully virtualize FreeNAS. We even 
include the guest VM tools in FreeNAS for VMware, mainly because we use VMware to do a lot 
of FreeNAS development. However if you have problems, there are no developer assets running 
FreeNAS as a production VM and help will be hard to come by. For this reason, | highly recom- 
mend that FreeNAS be run “On the Metal” as the only OS on dedicated hardware. 
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Has the technology sector finally slid into 
the realm of used car salesmen, lawyers 
and ambulance chasers? 


Adobe Photoshop inventor Thomas Knoll has made a call for the 
ethical use of the product. Has the technology sector finally slid into 
the realm of used car salesmen, lawyers and ambulance chasers? 


has been a discernible slide from grace over the 

years as to how IT departments, support staff etc. 
have been perceived both by society and management. 
While technology itself may be a factor in this, my per- 
sonal theory is not that technologists are any less mor- 
al per se than any other professional sector, but rather 
when people fear what they do not understand there is an 
instinctive impulse to demonise, categorise, pigeon-hole 
and control. Culturally, we may laugh at the classifications 
of “Geek” and “Nerd”, but ultimately outside of the techni- 
cal community, these are terms of insult rather than en- 
dearment — designed to put the target firmly in their place. 
None is a more hypocritical sight than an HR department 
spewing forth never-ending drivel concerning equality, 
diversity and fairness yet at the same time categorising 
staff in IT as “washing machine engineers”. Apparently, 
IT is now such a demystified specialisation, anyone with 
a screwdriver, a box cutter and a hammer (What — you 
don’t use hammers when upgrading a server?) is suffi- 
ciently competent. Sadly, this devaluation of skills and val- 
ue has permeated management — both middle and senior. 


N\ SA, security and social media ethics aside, there 


Maybe it is my non-cynical side coming out, but | don't 
believe for a moment that money alone is the main driver 
for career or indeed personal satisfaction for the major- 
ity of people. | would continue to work in IT even | wasn’t 
paid — provided my family and myself had a roof over 
our heads and food in our belly. The lie that we must be 
continual consumers has finally died along with the myth 
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that bankers have integrity, poverty is good for the soul 
and that the Western world we live in is a democracy. 
What drives people is a social contract, that they add val- 
ue by the works of their hands and in return from their 
employer, get little bits of paper or lumps of metal that 
they can exchange for whatever they want. The old model 
stated that we might not give you so many tokens now, but 
we will give you a decent pension, flexible working condi- 
tions, and maybe other benefits such as more holidays 
per year and job stability. This fitted well with a socially 
co-operative model, where the essential servants of so- 
ciety (e.g. Law enforcement, Nurses, Civil servants etc.) 
found a good deal of job satisfaction. If you want more 
benefits, move to a more corporately aggressive sector, 
but in turn you will be expected to compromise your pro- 
fessional integrity more, perform a job you hate, spend 
less time with your family, or just do something morally 
repugnant and hope that your conscience doesn't hurt too 
much. And of course, all with the added benefit of little 
or no job security. The downward spiral of rising costs, 
increased competition and the decreased buying power 
of salaries is finally hitting home across the profession- 
al, skilled and semi-skilled marketplace. The old model 
is dead. The new model demands the commercial stance 
of using technology to cut costs and bring efficiencies and 
it brings with it a very two edged sword. The life of a well- 
designed enterprise scale system can be measured in 
decades, and once the developers, designers, analysts 
and programmers have left, provided the business model 
of the corporate entity does not change much, the return 
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on investment can be huge and the creatives and others 
scaled down. So, in parallel with the false perception that 
anyone can do IT well now that it has become commodi- 
tised, we are heading towards another industrial revolu- 
tion driven by excesses of the military industrial complex. 
Far from learning from the past and using technology to 
bring ethical, moral and social benefit, corporate culture 
has bitten the very hand that feeds it, turned, and start- 
ed to consume its own children. To hell with the human 
and moral consequences, profit and efficiency is all that 
counts and it matters not how many people are put on the 
scrapheap in the process — be they skilled or unskilled. 
And we wonder why the alarm bells are going off by Bill 
Gates, Stephen Hawking and Elon Musk about the dan- 
gers of Artificial Intelligence. 


| don't believe that any creation can be greater than 
its creator. That does not mean that any creation in the 
hands of men cannot pose a significant threat to society. 
In the old days, programmers regularly put back doors in 
their software for maintenance purposes. The kill switch 
was always there. The danger is when we get to the 
stage that neither the end user nor the designer can ef- 
fectively hit the kill switch and this authority is delegat- 
ed to “the system” — and by that | mean either cultural, 
political or a hybrid mix of hardware and software. Case 
in point, we take the mobile phone network for granted 
but in a time of local emergency this can be dedicated to 
Law enforcement etc. cutting off the average consumer. 
While | don't have a problem with this particular scenario 
itself, what would happen if this control was exploited for 
political or economic purposes? Indeed this is happening 
now, but not by the technologists. The HR departments 
first port of call when hiring is Facebook, Twitter or Linke- 
din. All well and good you might think, but what about de- 
cisions being made about your credit worthiness based 
on your associates or lifestyle? Your insurance company? 
The robotic trawling of these sites is a feature of the cor- 
porate landscape, from HR to marketing and reputation 
management. It is not the technology or the technologists, 
it is the power behind the throne, the men in grey suits, the 
hand behind the curtain that truly drives the agenda — and 
those unaccountable faceless corporate clones that either 
wittingly or unwittingly go along with the agenda. 


The irony is that most IT professionals | have encoun- 
tered are a decent bunch whose heart is to improve things 
and make life better, more interesting, more fun. Unfortu- 
nately, we tend to be mesmerised by the environment we 
work in, and forget that outside the IT suite there are those 
— unlike the machines and systems we work with — who 
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have rather dark and ulterior motives. As our culture and 
society becomes more and more globalised — yet ironical- 
ly at the same time more compartmentalised — it is difficult 
to see this as the layers of management and ethical re- 
sponsibility are diluted. Nevertheless, by association, we 
are being tarred with the same brush. Too often, the poli- 
ticians, leaders and the establishment have echoed the 
benefits of change, automation, technology and progress 
but the end result has been far from the idyllic. For the UK, 
in the 80’s we moved to a service rather than a manufac- 
turing based economy. 35 years later there are still areas 
that are blighted by unemployment, especially amongst 
the youth. Be it the Chicago motor industry, shipbuild- 
ing in Glasgow or farming in France, the economic winds 
of change blow, but there remains a fundamental discon- 
nect between the vision, the implementation and the con- 
sequences. 


As technologists we must come to face the harsh truth 
that we fall into the same category as gunsmiths. What we 
design is not bad in itself, it is how it is utilised that mat- 
ters. Sadly, more than ever we better make sure that as 
a profession we are not exploited as useful idiots by those 
that wish to use our creative talents for evil and not for 
good. We urgently need to embrace a strong moral ethic. 
For as history has shown, should we experience a revolu- 
tion on the scale that Gates, Hawking and Musk envisage, 
the creatives, intelligentsia and the useful idiots will be the 
first to face the wrath. 


Rob Somerville has been passionate about technology since his ear- 
ly teens. A keen advocate of open systems since the mid-eighties, he 
has worked in many corporate sectors including finance, automo- 
tive, airlines, government and media in a variety of roles from tech- 
nical support, system administrator, developer, systems integrator 
and IT manager. He has moved on from CP/M and nixie tubes but 
keeps a soldering iron handy just in case. 
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Finding Security Insights, Patterns, and Anomalies 


in Big Data. Simulations and Security Processes 


The primary tool that we will be using in this chapter 

about simulations is Arena, which is commercial software 
developed by Rockwell Automation. Arena is a powerful 
modeling and simulation software allowing a user to model 
and run simulation experiments. We will be using a fully 
functioning perpetual evaluation version, which is available 
for study and download at (http://www.arenasimulation. 
com/Tools_Resources_Download_Arena.aspx). 


is a Windows desktop application, when you start us- 
ing the program, you will see three regions on the main 
Arena window. Let us familiarize you with the three regions: 


et us get started with simulations. Since Arena 


¢ At the left-hand side of the main window, you will find 
the Project bar containing three tabs: basic process, 
report and navigate panel. In the Project bar, you will 
also find various “Arena modules” to be used when 
building a simulation model. We will discuss more 
about Arena modules in the latter part of this section. 
¢ At the right-hand side, you will find the Model window 
flowchart view to be the largest part of your screen be- 
cause it is your workspace where you will create models. 
You will be creating graphical models using flowcharts, 
images, animations, and other drawn elements. 
At the bottom part of the flowchart view, you will find 
the Model window spreadsheet view, which presents 
all the data associated with the model. 
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This chapter will provide a high-level overview of creat- 
ing simulations in Arena. There are three main steps 
to making a simulation in Arena: 


¢ Design and create a model, 

¢ Add data and parameters to a model, 
¢ Runa simulation, and 

¢ Analyze a simulation. 


Designing and Creating a Model 

Before we start in Arena, we first need to create a “con- 
ceptual model” for a scenario we will simulate. A con- 
ceptual model is how you think a process should work 
—this could be anything from you just drawing it out on 
a piece of paper or just thinking about it. 

Once you have a conceptual model, the next step is to build 
the model in the workspace using the “modules” in Arena. Mod- 
ules are the building blocks of a model. There are two kinds of 
modules: the flowchart modules and the data modules. 
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The flowchart modules illustrate the logic of your simula- 
tion. Some common flowchart modules found in the “Basic 
Process’ tab of the Project bar are the following elements: 
CREATE, PROCESS, DECIDE, DISPOSE, BATCH, SEP- 
ARATE, and ASSIGN and RECORD. To use these mod- 
ules, you simply drag the flowchart module needed into 
the model and then you connect the modules together in 
the Model window flowchart view. For example, if | were to 
create our conceptual model of the IT service desk ticket 
queue, it would look like this (Figure 1). 

As you see in our figure, we used the CREATE, PRO- 
CESS, and DISPOSE modules to illustrate the logic of the 
queue. Once a service desk ticket is created by the IT 

Department (CREATE module), it is processed by the IT 
Department (PROCESS module), and it is closed by the IT 
Department (DISPOSE module). A bit confused? Rest as- 
sured, we have a whole chapter about this and it will get clear- 
er as we take you step by step through an actual scenario. 

For now, we are starting with a three-process scenar- 
io to get you thinking about simulation. This quick start 
model is provided on the companion site for download. 
For now, just think of it as creating a flowchart of your sce- 
nario. If you have used Microsoft Visio before, you will be 
right at home. 


IT Service Ticket IT Closure 


IT Processing 


(Process) 


Reciept (Create) (Dispose) 


Figure 1. /7 service desk process 


Adding Data and Parameters to the Model 

After creating the flowchart, the next step is to add da- 
ta to each of the flowchart modules. You may assign 
the values for each module by double clicking on the 
modules in the model, which will open up a small dia- 
log window. For example, for the CREATE module, let 
us say tickets arrive at an average of five per hour. 
You would enter that value directly into the CREATE 
module. Additionally, let us say tickets are processed 
and resolved at an average rate of 30 min. You would 
assign this value into your PROCESS module. We will 
provide you with a more detailed walk-through on how 
to do this later in this chapter. 


Running the Simulation 

After the model is complete, all you need to do is to select 
“Go” from the Run menu or press F5. There are other pa- 
rameters that you may want to set up before running the 
simulation, such as the replication parameters where you 
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can set the simulation period. But for the purpose of this 
quick introduction, we will just run the simulation. 


Analyzing the Simulation 

Arena provides a variety of reports so that you may ana- 
lyze the simulation. You access the Reports panel from 
the Project bar. 


CASE STUDY 

There are a lot of interesting uses for simulations in securi- 
ty. One of them is evaluating the effect of security controls 
or mechanisms in your enterprise that otherwise would 
be difficult to recreate. For this chapter, let us put our- 
selves in the position of an Information Security Officer, 
who needs to evaluate different anti-virus (AV) e-mail se- 
curity gateway offerings. One of the main things you will 
be concerned about is performance of the e-mail gateway 
device. Since the device will be sitting in-line and process- 
ing network traffic, you would want to make sure that the 
e-mail gateway is able to handle the volume of e-mails 
coming into your organization. Since this device will also 
sit in front of your e-mail server, there is no convenient 
way to test how the different e-mail security gateways will 
perform. This is where simulations come into play. Simu- 
lations give us a way to predict how a certain scenario or 
situation will play out based on available data. Of course, 
it will not be the same as testing the real thing, but it will 
at least provide us an estimate so that we can make an 
informed decision (Table1). 

One of the first things we need for a simulation is data. 
Fortunately, in our scenario, a vendor (hereafter referred 
to as Vendor 1) provided us with a data set comparing 
its e-mail security gateway solution with products from 
other vendors (hereafter referred to as Vendor 2 and Ven- 
dor 3). You can download this data set from the book’s 
website. Next, we will explain how this data set will be used 
in our scenario. 


Table 1. Vendor Scenario Data 


Average (s) 0.177271963 0.669560187 0.569069159 

Test data (s) 0.0077 0.0119 0.5994 
0.0018 0.0201 0.5269 
0.0101 3.4405 0.4258 
0.0144 0.0701 0.5109 
0.0134 0.02 0.5619 
0.006 0.0119 0.5017 
0.1103 0.0012 0.4382 
0.0113 0.013 0.4346 
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0.0116 
0.0185 
0.0021 
0.0088 
0.0051 
0.0061 
0.0106 
0.0064 
0.01 
0.0128 
0.0113 
0.01 
0.0058 
0.0023 
0.0126 
0.0128 
0.006 
0.0064 
0.0088 
0.011 
0.0142 
0.0058 
0.0062 
0.0063 
0.014 
0.0946 
0.0011 
0.0073 
0.0089 
0.0111 
0.0081 
0.0114 
0.0096 
0.6305 
0.0113 
0.0059 
0.0102 
0.065 
0.0063 
0.0189 
0.9503 
0.0236 
0.0094 
0.0076 
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0.0161 
0.0157 
0.2894 
0.0089 
0.0056 
0.0067 
0.0206 
0.221 
0.025 
3.067 
1.098 
0.0158 
1.145 
0.112 
0.0146 
0.0098 
0.0201 
0.0139 
0.0066 
1.945 
0.8112 
0.855 
0.874 
1.589 
0.0203 
0.89 
0.0112 
2.547 
3.4003 
2.3314 
0.0158 
0.0144 
0.0204 
0.0061 
0.0105 
2.578 
0.95 
0.721 
3.3614 
0.3078 
3.3444 
0.0103 


0.00254 


1.067 


0.4988 
0.49 
0.4843 
0.4602 
0.4431 
1.4135 
0.4199 
0.4332 
0.4162 
0.4386 
0.4342 
0.4309 
0.4146 
0.4392 
0.4678 
0.4608 
0.4689 
0.481 
0.4449 
0.4312 
0.453 
1.2839 
0.445 
0.4275 
0.4517 
1.092 
0.5119 
0.5966 
1.2248 
0.4345 
0.5527 
0.4991 
0.4213 
1.3264 
0.4312 
0.4246 
0.4422 
1.4509 
0.478 
0.4121 
1.1532 
0.4589 
0.4124 
0.5074 


0.0057 
1.0007 
0.0061 
0.0113 
0.0094 
0.0061 
0.0088 
0.0054 
0.9407 
12.2007 
0.0035 
0.0028 
0.0042 
0.083 
0.0009 
0.0078 
0.0357 
0.0068 
0.0107 
0.0128 
0.0113 
0.9457 
0.0109 
0.0181 
0.0099 
0.0066 
0.0111 
0.0108 
0.0159 
0.0155 
0.0113 
0.0057 
0.0064 
0.0126 
0.0171 
0.0038 
0.0059 
0.0043 
0.0066 
0.0069 
0.01 
0.0064 
0.0119 
0.0113 


0.905 
3.4747 
0.0205 
0.013 
0.0018 
0.0101 
1.345 
0.0936 
3.7085 
3.4655 
1.523 
0.0202 
0.0147 
0.9678 
0.0059 
0.0211 
0.0496 
0.016 
0.0177 
1.8678 
0.013 
0.812 
0.0071 
1.78 
0.0102 
0.832 
0.0127 
0.0144 
0.0026 
0.0772 
0.0136 
0.0101 
0.0125 
0.0146 
0.042 
0.1454 
1.89 
0.0407 
0.8901 
0.8542 
0.0059 
1.956 
1.993 
1.432 


0.4509 
0.4639 
0.4729 
0.4343 
0.4359 
0.4761 
0.4594 
0.6192 
1.1916 
0.5122 
0.4097 
0.4422 
0.4585 
1.282 
1.4524 
0.5503 
0.7331 
0.4823 
0.4378 
0.4388 
0.4349 
0.9953 
0.4457 
0.4099 
0.4278 
0.4231 
0.4346 
0.4988 
1.4738 
0.4918 
0.4157 
0.4327 
0.5496 
0.4308 
0.4525 
0.6053 
0.4243 
0.7431 
0.4764 
0.4635 
0.522 
0.4802 
0.4333 
0.4343 
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0.0111 0.0127 0.4347 
0.0064 0.0125 0.5521 

0.065 0.711 1.2924 
0.0912 0.0056 0.5121 

0.0059 0.0107 0.4661 

0.0125 0.0124 0.4177 
0.0113 0.013 0.4157 
0.8998 1.9081 0.4626 
0.0059 0.0102 0.4304 
0.0184 1.145 0.4216 
0.0099 0.0144 0.5201 


Table 2. Vendor Processing Time — Overall Performance 


Vendor 1 0.177271963 
Vendor 2 0.669560187 
Vendor 3 0.569069159 


Vendor 1 ran malicious e-mails through its e-mail secu- 
rity gateway and computed how fast the gateways pro- 
cessed the malicious e-mails (e.g., how fast the malicious 
e-mails were detected). Since Vendor 1 provided the data, 
as expected in terms of average processing times, Ven- 
dor 1 had an extremely short processing time (Table 2). 

You may be asking yourself, how do we validate these 
numbers? Typically you would just take this data at face 
value and accept these numbers. However, what if you 
wanted to dive deeper to see if these are actually accu- 
rate for your organization’s situation? This is where the 
fun part starts because we can do this through simulation. 
Let us dive into Arena! 

First off, let us deconstruct our scenario. We need three 
components to start our simulation: 


e First, we need to create the e-mails; 

¢ Second, we need to create the ‘e-mail security gate- 
way to process these e-mails; and 

¢« Third, we need to create the inboxes that will receive 
the e-mails. 


Fortunately, creating all of these components is fairly easy 
to do in Arena. Let us start first by creating a stream of e- 
mails that will come into our organization. This can be do- 
ne by using the CREATE module (Figure 2). 

One of the most important things that we need to 
do for a simulation is to create objects that will flow 
through the simulation that we are creating. In our sce- 
nario, the objects flowing through the system are the e- 
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Figure 2. /nserting the CREATE module 


mails that will go through our security devices. In Are- 
na, these objects are known as “entities.” To be able to 
create entities, we need a CREATE module. 

To make a CREATE module, all you have to do is drag 
the icon named create from the left-hand Basic Process 
bar to your work area. Your work area should look similar 
to Figure 3 below. It still looks a little sparse right now but 
this is only our first step. 

Once you have added the CREATE module, the next 
step is to start configuring the attributes and properties for 
that module. To assign value to attributes or properties of 
the module, double click on the CREATE shape so that 
a dialog window appears, as shown in Figure 4. 

In the dialog box, assign any name describing the en- 
tity being created. In this case, we labeled the entities as 
external e-mails. Let us change the entity type to “E-mail” 


Figure 3. Using CREATE to create external e-mail entities 
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as well. We will also tell the simulation the average rate 
of e-mail arrival. The arrival of e-mails could be different 
for each organization. There are different ways of estimat- 
ing this information (i.e., looking through your logs), but 
for the purposes of this example, we shall assume that on 
average an e-mail arrives every second. We can do this 
by changing the following: 


¢ Type: Random (Expo) 
¢ Value: 1 

¢ Units: Seconds 

e Entities per Arrival: 1 
¢ Max Arrivals: Infinite 
¢ First Creation: 0.0. 


At this point, we have created entities for our simula- 
tion. This means that e-mails can now enter our system. 
But where will it go? Right now, nowhere. We need these 
e-mails to be processed, so we will need to create a pro- 
cess. This is done by dragging the PROCESS module 
from the left-hand navigation bar into the workplace as il- 
lustrated in Figure 5. 

Since the e-mails going through the system need to be 
processed by the AV gateway, we will pattern our process 
to our gateway. In our simulation, the PROCESS mod- 
ule will represent the AV gateway that will be processing 
the external e-mails. Similar to what we did with the CRE- 
ATE module, we will configure the attributes and proper- 
ties of the PROCESS module. Open the dialog box for the 
PROCESS module in the same way you did with the CRE- 
ATE module by double clicking on it. 
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First, let us assign a name for the process module. For 
this example, we will name it “Security Gateway Vendor.” 
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Figure 5. Adding the PROCESS module 


Next we will set the ACTION for this element. In the ac- 
tion drop down, we will select “Seize, Delay, Release” ac- 
tion. This means that when an e-mail arrives, it will wait 
until the resource becomes available and seize the re- 
source, it will wait for the service interval, and then release 
the resource. This is essentially how an e-mail gateway 
operates: before a gateway sends an e-mail to the inbox, 
it will seize, delay (because of processing), and then re- 
lease either to a user’s inbox or a quarantine. 

The “Delay” is an important value here in our simulation be- 
cause it is actually the processing time. Relative to our sce- 
nario, this is the length of time the security gateway takes to 
process an e-mail to find out whether it is malicious or not. 

Our next step is to customize our scenario. Since we 
have the vendor results on the average processing time, 
let us put the average processing value of Vendor 1 for 
this example. Your dialog box should look similar to Fig- 
ure 6 and would have the following parameters: 


¢ Name: Security Gateway Vendor 

¢ Type: Standard 

¢ Action: Seize Delay Release 

¢ Priority Medium 

¢ Resources: Resource, Resource 1,1 
¢ Delay Type: Constant 

¢ Units: Seconds 

¢ Allocation: Value Added 

¢ Report Statistics: Checked 

¢ Value: 0.1777271863. 


The next step is to create the resource for our security 
gateway. Since we are only going to be using one security 
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gateway,v we will only create one resource. This setting 
is important if you are simulating multiple appliances 
or, in other cases, multiple processors. At this point, for 
simplicity, we will only create one resource, which can be 
done by clicking the “Add” button, which is located next 
to the Resource box. Your Resource dialog box should 
have the following parameters: 


¢ Type: Resource 
¢ Resource: 1 
¢ Quantity: 1. 


As the last step in our PROCESS module, we need to 
ensure that the CREATE module and the PROCESS 
module are linked together. In our scenario, this ensures 
that the e-mails created by the CREATE module goes 
to the PROCESS module to be processed by our AV 
gateway (Figure 7). Typically, Arena does this automatically; 
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Figure 7. Updating the resource property of the PROCESS module 
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however, if it does not, click the Connect button in the upper 
toolbar of Arena to link both modules as seen in Figure 8. 
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Figure 8. Connect the CREATE module with the PROCESS module 


Finally, after processing, we need somewhere for the 
e-mails to go. This is where we use the DISPOSE module. 
Drag a DISPOSE module into your work area and label it 
as “Mailboxes.” Then, connect the PROCESS module with 
the DISPOSE module. This means that after processing, 
the e-mails go to the mailboxes. 

At this point, you are probably thinking that something is 
amiss with this scenario. Why would all processed e-mails 
go directly to the inboxes, right? You are absolutely right 
that something is amiss. For the sake of keeping our step- 
by-step tutorial simple, let us work with what we have for 
now. We will continue to expand on our scenario to make 
it more realistic. Your final simulation should look similar 
to the model in Figure 9. 

Now that we have our simulation model, we are ready to 
run our first simulation. Before running our simulation, you 
will need to configure the different settings for the simu- 
lation. Since a simulation is technically trying to recreate 
a real-world scenario, we need to set up how long and 
how frequently we would like to let the scenario run. 

This is fairly easy to do in Arena. Just click on Run (it is 
a selection on the top bar) and select Run Setup. For this 
simulation let us run it three times for 7 days, 24-hours 
a day. Since e-mails arrive at a one second interval, the 
base time unit will need to be changed to seconds. You will 
see a dialog box similar to Figure 10, in which you will add 
the following parameters: 


¢ Number of Replications: 3 


¢ Initialize Between Replications: Statistics and System 
Checked 
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¢ Warm-up Period: 0.0 

¢ Replication Length: 7 

¢ Hours Per Day: 24 

¢ Base Time Units: Seconds 

¢ Time Units: Hours 

¢ Time Units: Days 

¢ Termination Condition: Leave Blank. 
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Figure 9. Connect the CREATE module with the PROCESS module 


After the Run parameters have been configured, we will 
add some information on the Project Parameters to de- 
scribe the project by clicking on the Project Parameter tab 
(see below figure). We are now ready to run our simula- 
tion. You do this by simply clicking Run, then selecting Go. 


¢ Project Title: Security Gateway 
¢ Project Description: Add any description 


After clicking Go, the simulation will animate and there 
will be elements moving. You will see “e-mails” coming 
from the CREATE module (external e-mails), moving to 
the PROCESS module (security gateway), and being ac- 
cepted by the DISPOSE module (inbox) (Figure 11). 
Congratulations, you have now completed your first 
simulation! The simulation may take some time to process 
before we get the results. Unfortunately, even with setting 
at the highest speed, running three simulations on e-mails 
over a /-day period will take some time to process. 
Fortunately, Arena has a feature, which allows what is 
called “batch processing.” Batch processing bypasses all 
of the animation, which speeds up processing. To do this, 
you first need to stop the simulation by clicking on Run, 
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Figure 10. Run parameters set up 


and then End. Next, select Run Control and click on Batch 
Run (No Animation). By doing this, you will speed up the 
simulation so that you can generate the results faster (Fig- 
ure 12). 
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Figure 11. Project set up 


Let us try running the simulation again using these new 
parameters. You will notice this time that you see no ani- 
mation and you immediately receive the results. 

Your output will be a report, including some interesting 
values such as minimum averages, maximum averages, 
minimum values, and maximum values. Basically, these 
values are the descriptive statistics for your simulation 
processing times, which we ran three times in 7-day in- 
crements (Figure 13). 
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The real value of the simulation is seen once we start 
comparing the vendors. Let us try doing this next. For each 
vendor, change the delay value to match each vendor’s 
average processing time. If you gather the results, you 
should obtain the results in Table 3. Your results should 
show that Vendor 1’s claims are accurate: on average, 
Vendor 1 shows the best performance. 

For now, we will accept that Vendor 1's claim is correct. 
However, as you know, statistics can sometimes be interpret- 
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ed to provide misleading information. Let us start by going 
back and look at the original data that the vendor gave us. 

A very interesting point about the original data is that 
the vendor provided the actual results of their testing. 
With the results for individual processing times for each 
of the e-mails, instead of just the average processing time 
for all of the e-mails, we can use a very simple yet relative- 
ly well-known technique called standard deviation (SD). 

SD shows the variation or dispersion that exists in rela- 
tion to the mean (also called average). A low SD indicates 
the data points tend to be close to the mean (also called 
expected value); a high SD indicates the data points are 
spread out over a larger range of values. 

Let us start with some simple spreadsheet work. 
We want to open the file containing the sample data 
and obtain the SD of the data by using the STDEVP 
function (=STDEVP). This explanation may be somewhat 


Vendor 1 Vendor 2 Vendor 3 
Average (sec) 0.177271963 0.669560187 0.569069159 
Standard Deviation |=STDEVP(D5:D111' 0.274832835 
= Test Data (sec) 0.0119 0.5994 
Petes Peo Se ee ae Seo 0.0201 0.5269 
Figure 12. A running simulation 3.4405 0.4258 
0.0701 0.5109 
i a ger rican 0.02 0.5619 
nedien ant =< |e ee) a c |Me 0.0119 0.5017 
reo UUs ele lesieseelaees) 0.0012 0.4382 
—— | 0.013 0.4346 
Sut rer shiners 0.0161 0.4988 
Soro 0.0157 0.49 
I = 0.2894 0.4843 
2 0.0089 0.4602 
=A 0.0056 0.4431 
- 0.0067 1.4135 
= 0.0206 0.4199 
0.221 0.4332 
paisa 0.025 0.4162 
3.067 0.4386 
ou 1.098 0.4342 
0.0158 0.4309 
1.145 0.4146 
0.112 0.4392 
0.0146 0.4678 
0.0098 0.4608 
ee 0.0201 0.4689 
_ 0.0139 0.481 
| 0.0066 0.4449 
Figure 13. Doing a batch run Figure 14. Computing SD in a spreadsheet 
Table 3. Vendor Processing Time — Initial Simulation Run 
Vendor 1 0.177271963 0.01911149 0.01897183 0.01919783 0.00 0.9207 
Vendor 2 0.669560187 0.6809 0.6760 0.6863 0.00 11.6437 
Vendor 3 0.569069159 0.3770 0.3740 0.3740 0.00 8.3927 
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confusing, so see Figure 14. Open the next tab of the 
sample data containing the computation. 


Table 4. Vendor Processing Time — Adding SD 


Vendor 1 0.177271963 1.185744915 
Vendor 2 0.669560187 1.026043254 
Vendor 3 0.569069159 0.274832835 
Average (s) 0.177271963 0.669560187 0.569069159 
Standard 1.185744915 1.026043254 0.274832835 
deviation 


After computing the SD for all of the vendors, we see 
that Vendor 1 actually has a big SD. This means that the 
results of the test data vary greatly. For example, it pro- 
cesses e-mails very fast in some cases, but in other cas- 
es, it processes e-mails very slowly. You may be asking 
yourself, what exactly does this tell us? Obviously, those 
of you who understand SDs probably already have an in- 
kling of what this means, but we will run a simulation so 
we can see what our scenario generates (Table 4). 

Now, we will go back to the simulation to enter our newly 
computed values. Click on the PROCESS module. How- 
ever, let us change things up a bit. Instead of using the 
“Constant” delay type, we will use the “Normal” delay type 
or what we call a normal distribution. The normal distri- 
bution is a function telling you the probability that an ob- 
servation, in some context, will fall between any two real 
numbers. 

In the PROCESS dialog box, we will maintain the mean 
value, but we will now add the SD for the vendor. The en- 
tries you select should be similar to the below values, 
which will be put in the dialog box (See Figure 15). Next, 
we will run the simulation. 


¢ Name: Security Gateway Vendor 
¢ Type: Standard 

¢ Action: Seize Delay Release 

¢ Priority: Medium 

¢ Resource: Resource, Resource 1,1 
¢ Delay Type: Normal 

e Units: Seconds 

¢ Allocation: Value Added 

¢ Value: 0.177271963 

¢ Std Dev: 1.185744915 

¢ Report Statistics: Checked. 


We will run the simulation for all the vendors and collect 
their results. Remember to make the change to “Normal” 
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for all of the vendors and to add in the SD. Once you 
have run everything and collected the results, your re- 
sults should be similar to values in the tables below. 


Hate eet - cae ite tte 
F J 


Crh FP BA tae Go 
°° 2D Po wl: Beas |e lee 
Dena a 8 


- ee Po ree of ii 


merBrm-+Be ws FP Bae BE ates 


—— 
& faa Procene 


Figure 15. Updating the PROCESS Dialog’s standard deviation 


You should now notice that the results are quite inter- 
esting. The values have changed a considerable amount 
because we used the normal distribution. In fact, Vendor 
1 did not perform as well as expected with these results. 
In this scenario, Vendor 3 actually had better results. 

The reason for this is that Vendor 3 had more consis- 
tent results. The processing times for Vendor 3 were more 
stable and, more importantly, less variable. Conversely, 
Vendor 1 had a lot of variability, which greatly affected 
the overall processing times. This is why it is important to 
understand what you are processing and how it will affect 
your results. Had you just gone with the vendor results, 
you would not have known this information — this provides 
you with value-added information, which could affect your 
choice of a vendor (Table 5). 

For the final part of our tutorial, we will extend our simu- 
lation model to make it more detailed and realistic. In the 
previous scenario, we assumed that all e-mails were ma- 
licious, but in reality we would never do this. For a more 
realistic scenario, we will incorporate the DECIDE module 
to create conditional branches. 

The DECIDE module can be found in the Basic Process 
tab, which is located on the left-hand side of our work ar- 
ea. The DECIDE module helps us to create conditions (al- 
so known as “if-then” conditions) that are similar to what 
you would see in a flowchart (Figure 16). 

We will now create a scenario with conditional 
elements. As we already mentioned, not all e-mails will 
have malicious attachments. Let us say only 5% of all 
e-mails will have malicious attachments. How did we 
get 5%? This is entirely dependent on you, but to have 
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Table 5. Vendor Processing Time — Additional SD Settings 


Vendor 1 1.0290 1.0231 1.0342 0.00 25.1238 
Vendor 2 3.8393 3.8183 3.8531 0.00 46.5414 
Vendor 3 0.4661 0.4650 0.4680 0.00 9.4981 
Vendor 1 0.01911149 0.01897183 0.01919783 0.00 0.9207 
Vendor 2 0.6809 0.6760 0.6863 0.00 11.6437 
Vendor 3 0.3770 0.3740 0.3740 0.00 8.3927 


a more realistic scenario, you should probably try to 
check industry benchmarks. For example, Symantec 
issues a monthly intelligence report similar to the one 
in this link where you can find benchmarks: http://www. 
symantec.com/content/en/us/enterprise/other_resources/ 
b-intelligence_report_07-2014.en-us. pdf. 
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¢ Name: Malicious? 
¢ Type: 2-way by Chance 
¢ Percent True (O—100): 5%. 


You should be familiar with the RECORD module, which 
acts like an advance counter. This module is used to run 
different computations and to store the processed re- 
sults within the module. For this scenario, let us make 
a simple counter using the RECORD module to track 
clean and malicious e-mails. If an e-mail is malicious, 
then we will assume the action to be taken is quaran- 
tining the e-mail. If the e-mail is clean, the action to be 
taken is to send it to the user’s inbox. Therefore, we will 
make two counters: one is a Quarantine counter and the 
other is a Mailbox counter. 
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Figure 16. Adding a DECIDE module in the simulation =a 
Now, let us go back to our workspace. Drag a DE- |= & 
CIDE module into the work area, and double click on the =" 
module. Once you are at the dialog box, type in a name a = 
and change the Type to “2-way chance,” which is the - 


default. Since there’s a 5% chance of a mail being ma- 
licious, you will enter 5% in the Percent True text box. 
Your entries should be similar to the parameters shown 
in Figure 17. 

Finally, we will close of the system by adding a DIS- 
POSE module for both the True and False branches. Note 
that all simulations must have a DISPOSE module. Let us 
label the DISPOSE modules as Quarantine for True then 
Mailbox for false. This will be a little understandable when 
we Start talking about counters. 
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Figure 17. Updating the properties of a DECIDE module 


We do this by connecting the RECORD modules in- 
to the DECIDE module, similar to Figure 18. As with all 
simulations, all paths should have an end point. So, you 
need to remember to create DISPOSE modules for the 
two paths: one for the clean e-mails and one for the mali- 
cious e-mails. 
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We will now configure the parameters of our RECORD 
module. Double click on each of the RECORD modules 
and set it to “Count” with a value of 1. This means that if 
the e-mail was malicious, then the RECORD module for 
the Quarantine RECORD module will be increased by 1. 
You would do the same for the Mailbox RECORD module. 
In the context of this scenario, the following applies: if the 
e-mail is malicious (therefore, YES), then the counter for 
malicious e-mails will be incremented by 1. If the e-mail is 
not malicious, then the clean e-mail counter will be incre- 
mented by 1 (Figure 19). 


¢ Name: Malicious E-mail 

¢ Type: Count 

¢ Value: 1 

¢ Record into Set: Unchecked 

¢ Counter Name: Malicious E-mail. 


That is it — our simulation is now complete! We have cre- 
ated a more complex simulation, which utilized the DE- 
CISION and RECORD modules. All you have to do now 
is to run the simulation and wait for the reports to be 
generated (Figure 20). 
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Figure 18. Creating RECORD and DISPOSE modules 


As you can see, our simulation is slowly getting more 
advanced. However, we are still not finished with it. What 
about efficacy? The vendor actually provided us with 
efficacy information and we can use this information to 
improve our simulation. The question is how do we incor- 
porate this information into our simulation (Figure 21)? 

In our previous simulation, we assumed that all e-mails 
that were considered clean were actually clean but in reality, 
things seldom work this way because malicious e-mail will 
get through AV checks. This is why the vendor provided us 
with the ratings of efficacy for each of the products being 
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Figure 19. Updating the properties of the RECORD module 


reviewed. Next, we need to add another conditional element 
in the simulation so that we may include this process. 

We will add another DECIDE module to the second filter 
for the clean decision, but this time, we will add a condi- 
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Figure 20. Viewing the report 
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Figure 21. Additional report information 
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Figure 22. Removing false negatives through another DECISION 
module 


tion for every clean e-mail. In this updated scenario, once 
the decision is made that an e-mail is clean, we place an- 
other decision regarding “how sure are we that the e-mail 
is clean.” We will call this our “True Clean” decision box, 
which is a layer to show the probability that a clean e-mail 
is actually clean. By adding this decision box, we are able 
to provide a means to determine “false negatives” or mali- 
cious e-mails that were missed by the security gateway. 
Your updated simulation should look similar to Figure 22. 


Table 6. Vendor Processing Time — Including Efficacy 


99.90% 


We will now configure our new DECISION module. Dou- 
ble click the True Clean box and add the efficacy rating 
into the Percent True box. This will simulate the probability 
that the e-mail is actually clean. We then add a counter to 
“how many e-mails that were considered clean were actu- 
ally malicious.” We will use the RECORD module to add 
a “Missed Malicious E-mail” box. Below are the e-mails 
that the AV missed, where the vendor’s verdict was clean 
but the e-mails were actually malicious (Table 6). 

Using our vendor spreadsheet, if the vendor’s security 
gateway has a 99.9% efficacy then we put 99.9% in the 
Percent True. These values will allow us to compute the 
probability of an e-mail actually being clean which equates 
to the efficacy in our simulation. See Figure 23. 


¢ Name: True Clean 


¢ Type: 2-way by Chance 
¢ Percent True (O—100): 99%. 
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Finally, let us run our simulation and wait for our report! 
We will do this for all of our vendors. Remember to make 
changes to the average processing times, the SD and 
the efficacy of each simulation. 

As we wrap up this chapter, let us go through the com- 
pleted simulation statistics in Figure 24. In summary, here 
are the observations we obtained from our simulation. 
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Figure 23. Adding efficacy into the simulation 
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Figure 24. Viewing the final report 


is actually pretty good in terms of 
performance. When we start looking at the efficacy 
(.e., 99.9% vs 98%) and considering the amount 
of e-mails processed in a week, the difference 
between a 99.9% efficacy and a 98% efficacy rate is 
a staggering amount. The difference can be as large 
as 8000 malicious e-mails! 


¢ Even a 99.9% efficacy would result in 568 malicious 


e-mails, which is still a lot of malicious e-mails. This 
shows that even when a vendor’s AV is used, there is 
still a big chance that one of your employees could be 
infected. 
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Vendor1 1.0290 1.0231 1.0342 0.00 25.1238 604,918 568.33 
Vendor2 3.8393 3.8183 3.8531 0.00 46.5414 605,514 1704.00 
Vendor3 0.4661 0.4650 0.4680 0.00 9.4981 605,311 9284.00 


The following tables provide a summary of the results we _ virus propagation within a network to see how fast it will 


collected during the simulation: affect your enterprise. You could also use simulations to 
see the effects of patching, of re-imaging machines and 

Average Processing Times of AV updates. On a larger scale, simulations could be 
Average Processing used to demonstrate cyber attacks against your organiza- 

Vendor 1 0.177271963 tion. You can create a simulation representing your whole 

Vander2 0.669560187 network, including firewalls, a intrusion prevention sys- 


tem and network segments, to see how attacks would or 

would not be detected, among other things. In conclusion, 
In this chapter, we demonstrated how it is possible to simulations in security are particularly useful in evaluat- 

simulate performance when it is difficult to testasystemor ing the effect of security controls or mechanisms in your 

otherwise obtain results. In our security scenario, we sim- —_ enterprise that would, otherwise, be difficult to recreate. 

ulated an AV gateway for three vendors; however, there 

are a lot of other interesting uses for simulations. Another 

possible use of simulations in security could be recreating 


Vendor 3 0.569069159 


Using a Constant Delay Type 


Vendor 1 0.01911149 0.01897183 0.01919783 0 0.9207 
Vendor 2 0.6809 0.676 0.6863 0 11.6437 
Vendor 3 0.377 0.374 0.374 0 8.3927 


Using a Normal Distribution (STD) 


Vendor 1 0.17727196 1.18574492 

Vendor 2 0.66956019 1.02604325 

Vendor 3 0.56906916 0.27483284 

Vendor 1 1.029 1.0231 1.0342 0 25.1238 

Vendor 2 3.8393 3.8183 3.8531 0 46.5414 

Vendor 3 0.4661 0.465 0.468 0 9.4981 
Final Results 

Vendor 1 1.029 1.0231 1.0342 0 25.1238 568.33 

Vendor 2 3.8393 3.8183 3.8531 0 46.5414 1704 

Vendor 3 0.4661 0.465 0.468 0 9.4981 9284 
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Data Used in the Simulation 


Average (s) 
Standard deviation 
Test data (s) 
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0.177271963 
1.185744915 


0.0077 
0.0018 
0.0101 
0.0144 
0.0134 
0.006 
0.1103 
0.0113 
0.0116 
0.0185 
0.0021 
0.0088 
0.0051 
0.0061 
0.0106 
0.0064 
0.01 
0.0128 
0.0113 
0.01 
0.0058 
0.0023 
0.0126 
0.0128 
0.006 
0.0064 
0.0088 
0.011 
0.0142 
0.0058 
0.0062 
0.0063 
0.014 
0.0946 
0.0011 
0.0073 
0.0089 
0.0111 
0.0081 
0.0114 


0.669560187 
1.026043254 0.274832835 
0.0119 0.5994 
0.0201 0.5269 
3.4405 0.4258 
0.0701 0.5109 
0.02 0.5619 
0.0119 0.5017 
0.0012 0.4382 
0.013 0.4346 
0.0161 0.4988 
0.0157 0.49 
0.2894 0.4843 
0.0089 0.4602 
0.0056 0.4431 
0.0067 1.4135 
0.0206 0.4199 
0.221 0.4332 
0.025 0.4162 
3.067 0.4386 
1.098 0.4342 
0.0158 0.4309 
1.145 0.4146 
0.112 0.4392 
0.0146 0.4678 
0.0098 0.4608 
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0.0113 0.0136 0.4157 of experience as a leader of research and analytics teams, specializing in 
0.0057 0.0101 0.4327 predictive modeling, simulations, econometric analysis, and applied sta- 
0.0064 0.0125 0.5496 tistics. Robert works with a team of researchers who utilize simulation 
0.0126 0.0146 0.4308 and big data methods to model the impact of catastrophes on millions of 
insurance policies...simulating up to 100,000 years of hurricanes, earth- 
0.0171 0.042 0.4525 
quakes, and wildfires, as well as severe winter and summer storms, on 
pause Oa 0.6053 more than 2 trillion dollars worth of insured property value. He has used 
0.0059 1.89 0.4243 predictive modeling and advanced statistical methods to develop auto- 
0.0043 0.0407 0.7431 mated outlier detection methods, build automated underwriting models, 
0.0066 0.8901 0.4764 perform product and customer segmentation analysis, and design com- 
0.0069 0.8542 0.4635 petitor war game simulations. Robert has a master’s degree in Informa- 
0.01 0.0059 0.522 tion Management from the Harvard University Extension. 
0.0119 1.993 0.4333 
0.0113 1.432 0.4343 |. Miyamoto is a computer investigator in a government agency with 
0.0111 0.0127 0.4347 over 16 years of computer investigative and forensics experience, and 
0.0064 0.0125 0.5521 12 years of intelligence analysis experience. |. Miyamoto is in the process 
0.065 0.711 1.2924 of completing a PhD in Systems Engineering and possesses the follow- 
| ) ing degrees: BS in Software Engineering, MA in National Security and 
0.0912 0.0056 0.5121 ; : ; 
Strategic Studies, MS in Strategic Intelligence, and EdD in Education. 
0.0059 0.0107 0.4661 
0.0113 0.013 0.4157 
0.8998 1.9081 0.4626 Jason L. Martin is Vice President of Cloud Business for FireEye Inc., 
0.0059 0.0102 0.4304 the global leader in advanced threat-detection technology. Pri- 
or to joining FireEye, Jason was the President and CEO of Secure 
0.0184 1.145 0.4216 
DNA (acquired by FireEye), a company that provided innovative se- 
0.0099 0.0144 0.5201 


Robert McPherson leads a team of data scientists for a Fortune 100 Insur- 
ance and Financial Service company in the United States. He has 14 years 


curity products and solutions to companies throughout Asia-Pacif- 


ic and the U.S. Mainland. Customers included Fortune 1000 compa- 
nies, global government agencies, state and local governments, and 
private organizations of all sizes. He has over 15 years of experience 
in Information Security, isa published author and speaker, and is the 
cofounder of the Shakacon Security Conference. 


Information Security Analytics: Finding 
Security Insights, Patterns, and Anomalies 


in Big Data 


ANALYTICS 


by Mark Ryan M. Talabis, Robert McPherson, Inez Miyamoto and Jason L. Martin 


This book provides insights into the practice of analytics and, more importantly, how readers 
can utilize analytic techniques to identify trends and outliers that may not be possible to 
identify using traditional security analysis techniques. It contains information on open-source 
analytics and statistical packages, tools, and applications, as well as step-by-step guidance 

on how to use analytics tools and how they map to the techniques and scenarios provided. 
Readers learn how to design and utilize simulations for «what-if» scenarios to simulate security 
events and processes, and how to utilize big data techniques to assist in incident response and 
intrusion analysis. Written by security practitioners, for security practitioners, the book includes 
real-world_case studies and scenarios for each analytics technique. 


—hitp://store elsevier.com/ 


Using FreeBSD as a File Server with ZFS 


Ivan Voras 


The ZFS storage workshop will teach you how to create a ZFS file system from scratch and build a file server on top 
of it, but it will also teach you how ZFS, file systems and storage servers work in general. You will learn what ZFS 
looks like, its many features and quirks, and how to use it in a FreeBSD server as a building block of a small file 
server. 


ZFS is the ground-breaking file system originally developed at Sun Inc. for their Solaris operating system. It was 
open-sourced as a part of their OpenSolaris initiative and from there has spread to multiple other operating systems. 
FreeBSD was the first one to implement a working port, and though it has taken a fairly long time of tweaking and 
Stabilization, it is now a robust and popular choice. There are products which successfully build upon the technolo- 
gies of FreeBSD and ZFS, such as FreeNAS and its related enterprise-class products from iXsystems, which au- 
tomate and simplify a lot of the tasks, but all of them use the same ZFS interface under the hood, which is not that 
complicated in itself. 


The requirements for this workshop are decent knowledge of FreeBSD, a basic familiarity with command-line op- 
erations, and a system (possibly a virtual machine) on which the student will perform the required tasks, containing 
at least four hard drives (physical or virtual). Since the topic of this workshop is file servers, the participants must 
prepare a virtual or a physical machine with at least two disk drives (and preferably 4), which which to perform the 
exercises and the setup from the workshop. 


http://osdmag.org/course/using-freebsd-as-a-file-server-with-zfs-2/ 


Ivan Voras is a FreeBSD developer and a long-time user, starting with FreeBSD 4.3 and throughout all the versions since. 
In real life he is a researcher, system administrator and a developer, as opportunity presents itself, with a wide range of 
experience from hardware hacking to cloud computing. He is currently employed at the University of Zagreb Faculty of 
Electrical Engineering and Eomputing and lives in Zagreb, Croatia. You can follow him on his blog in English at http:// 
ivoras.net/blog or in Croatian at http://hrblog.ivoras.net/, as well as Google+ at https://plus.google.com/+IvanVoras. 


Our courses are available online in Premium Membership. 
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3 easy steps to optimized checkouts: 


With Gate2Shop, you can optimize An effective payment page variant With dozens of alternative and local 
your payment pages by using testing tool, A/B Testing helps you payment methods offered in 
ready-made templates or by gain insight into user behaviour, multiple currencies, the personal- 
customizing payment pages to your increase payment conversion in the ized checkout allows you to reach 
site look and feel. short and long term. users from all around the world. 


wW Easy integration wW Cross-platform ewWSecure 
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Sell. More. 


Call for a free consultation: +44 20 3051 0330 
WWw.g2s.com 


